Building 3D Interactive Narratives with Large Language Models: A Development Guide

1. From a Single Prompt to an Experienceable Artifact

Asking a Large Language Model (LLM) to complete an “ambitious” creative task might sound as simple as writing a prompt, but it actually involves a comprehensive pipeline spanning narrative design, visual expression, interaction logic, and engineering implementation. This article uses a specific task as an example: creating an interactive website that explains to general audiences how a classic sculpture is born from a block of stone. The goal is to leave viewers with a new understanding of “carving” after the experience, rather than just memorizing a few facts.

The value of this type of task lies in its “non-standard” nature. It is not a checklist-style page with a single correct implementation path. Instead, it tests whether the model can make autonomous judgments, drive the project forward, and ultimately deliver a runnable result.围绕这一目标, we can break the entire process down into six phases: requirement abstraction, environment assessment, narrative modeling, asset processing, interaction implementation, and acceptance testing.

2. Requirement Abstraction: Translating Creative Intent into Executable Constraints

The original requirement is a natural language description containing three layers: the goal (helping ordinary people understand the birth of a sculpture), the experience requirement (making the audience willing to stop and observe the work), and open authorization (the execution team decides the narrative angle, visual style, interaction method, and technical route). Engineering-wise, this must be translated into verifiable constraints:

• Goal Constraint: After completing the experience, the audience should be able to articulate at least one insight about “subtractive creation,” such as “the statue is not added; it is revealed by removing what is unnecessary.”
• Experience Constraint: The single-session experience duration should be controlled between 3 to 8 minutes, featuring at least one interactive node requiring active operation.
• Delivery Constraint: The output must be an accessible webpage containing real 3D models, interactive camera controls, and state changes. Static images or design documents are not accepted.
• Compliance Constraint: Assets must have clear authorization sources, and model assets must record their origin and license agreements.

This layer of translation is critical. Generative models excel at providing creative ideas in an open space but easily lose focus at the “delivery” step. Breaking down ambiguous intent into verifiable items preserves creative freedom while establishing an acceptance benchmark.

3. Environment Assessment: Confirm Capabilities Before Execution

Before diving in, it is necessary to assess the capability boundaries of the execution environment. Common assessment dimensions include:

1. Code Execution Capability: Whether a runnable frontend engineering environment is available, including the ability to install dependencies, start local services, and build the project.
2. Resource Retrieval Capability: The ability to search public materials and download 3D assets that comply with license agreements.
3. Rendering Capability: Browser WebGL support, and whether the vertex and polygon count of 3D models falls within an interactive range.
4. Iteration Capability: The ability to preview the output and fix defects practically, rather than judging the effect based solely on descriptions.

The assessment conclusion directly dictates the technical route. For instance, if a model’s polygon count is too high, mesh simplification must be performed before loading. If the runtime environment does not support certain advanced rendering features, a more stable lighting and material solution should be adopted. A common pitfall is starting directly with an “ideal solution” and discovering at the delivery stage that the environment doesn’t support it, forcing a retreat to a design document.

4. Narrative Modeling: Making “Subtraction” a Perceptible Structure

Centered around the theme of “a sculpture being born from stone,” two typical narrative structures can be extracted, corresponding to different interaction paradigms:

Structure 1: Process Metaphor. This uses “remaining form” as the core concept, understanding the finished product as what is left after the stone is removed. In implementation, this manifests as a dark space with a complete block of stone in the center. When the user holds down an action button, the stone peels away layer by layer, and the form gradually emerges. This structure emphasizes the physical rhythm of operation, suitable for conveying the concept that “creation is an irreversible trade-off.” A reverse operation entry point can be set, but clicking it triggers a prompt saying “there is no undo button for materials,” using the interaction itself to express the concept.

Structure 2: Exhibition Reader. This focuses on episodic narration, using a light-colored paper-like texture for the background. The process is divided into several acts, and a slider controls the stone as it is gradually reduced from a block to the finished product. This structure has a higher information density, suitable for carrying explanatory content like historical background, creation era, and material information. It has a low interaction cost and appeals to a broader audience.

These two structures are not mutually exclusive. Engineering-wise, they can be organized as two paths within the same site: one leaning towards experience and the other towards reading. Both share the same 3D assets and camera system, branching only in the state machine and UI layer. This controls implementation costs while covering audiences with different preferences.

5. Asset Processing: Engineering Points for Real Scanned Models on the Web

Regardless of the narrative structure used, the core asset is the 3D model of the sculpture. In practice, publicly licensed high-fidelity scanned models are typically used. The following engineering points should be noted:

Asset Integration Workflow (Schematic)
1. Obtain model files (e.g., STL / OBJ / GLTF)
2. Verify license agreements, record source and attribution info
3. Mesh cleanup: remove non-manifold faces, merge duplicate vertices
4. Polygon reduction: set triangle limits based on target devices (mobile recommended 50k - 150k)
5. Coordinate normalization: unify origin, orientation, and scale
6. Material and lighting adaptation: PBR parameters, ambient occlusion baking
7. Export to a web-friendly format (e.g., GLTF + Draco compression)
8. Loading strategy: chunked or progressive loading, load only low-poly model on the first screen

Among these, polygon reduction and loading strategy have the most significant impact on user experience. High-fidelity scanned models often have millions of polygons, and loading them directly will cause first-screen stuttering or even crashes. A viable approach is to prepare two levels of assets: a low-poly model for the first screen and quick preview, and a high-poly model loaded asynchronously after the user enters the detail observation phase. Meanwhile, the stone peeling animation should not rely on per-face deletion. Instead, it should be implemented via clipping planes, opacity gradients, or hierarchical mesh replacement to maintain a stable frame rate.

6. Interaction and Engineering Implementation: The State Machine is Core

The complexity of such sites is concentrated in state management. Every user operation changes the state across three dimensions: “remaining material,” “current narrative act,” and “camera position.” These three must remain consistent. Using an explicit state machine to describe this is recommended:

// State definitions (Schematic)
const states = {
  PROLOGUE:   { progress: 0.0, camera: 'wide',   ui: 'intro'    },
  CHISELING:  { progress: 0.0, camera: 'orbit',  ui: 'action'   },
  REVEAL:     { progress: 0.7, camera: 'detail', ui: 'observe'  },
  EPILOGUE:   { progress: 1.0, camera: 'hero',   ui: 'summary'  }
};
// Progress-driven geometry representation
function applyProgress(model, p) {
  // p ∈ [0,1], mapped to clipping plane position or hierarchical visibility
  model.clipPlane.constant = lerp(model.bounds.min, model.bounds.max, p);
  model.material.opacity = smoothstep(0.0, 0.9, p);
}

There are three key design principles: Single source of progress—all visual representations are derived from the same progress value to avoid conflicting states; Operations are reversible but semantics are irreversible—allow users to look back, but copy and prompts should continuously emphasize the irreversibility of material removal; Performance fallback—automatically reduce model precision and shadow quality on low-end devices to ensure smooth interaction takes priority over visual fidelity.

7. Video Derivative: Compressing Interactive Experience into a Narrative Short Film

After completing the website, a common extension is to produce a 60 to 90-second short film to introduce the essence of the work to the public and spark interest. The expression logic of a vertical short video is completely different from that of a webpage and requires reorganization:

• Opening 3 seconds: Directly present the visual hook of “a human form faintly emerging from the stone” rather than establishing background.
• Middle structure: Use a complete “removal-reveal” process as the main line, accompanied by a core concept subtitle, avoiding information overload.
• Ending guidance: Freeze on the full view of the artwork and prompt the user to enter the interactive page to experience it themselves.
• Engineering delivery: Output a playable vertical video file while retaining the editable project and asset source list for future modifications.

In practice, the most error-prone aspect of the video phase is pacing and audio-visual alignment. It is recommended to either set the edit points first and then do the voiceover, or create the voiceover timeline first and then fill in the visuals. Choose one approach and avoid bi-directional反复 adjustments.

8. Practical Findings and Experience Summary

Through multiple similar practices, several universal experiences can be summarized:

1. Creative capability and delivery capability are two different things. Generative models often provide high-quality concepts at the design level but easily lose focus midway through long-pipeline tasks, stopping at “looks good” semi-finished products. The solution is to slice the task into smaller stages with acceptance checkpoints, requiring each stage to be runnable and previewable.
2. Cost and latency are invisible thresholds for usability. Long-pipeline creative tasks involve massive code generation and debugging. API call costs and response latency will significantly affect iteration counts. Choosing a lower-cost inference tier combined with clearer阶段性 goals is often more sustainable than blindly pursuing the most powerful model tier.
3. Real assets are the cornerstone of experience. Using clearly licensed real scanned models builds more credibility than procedurally generated alternatives, but it must be accompanied by a complete asset optimization workflow.
4. Acceptance must be hands-on. Actually opening the page, playing the video, and testing on low-end devices is the only reliable way to discover layout misalignments, loading failures, and audio-visual desync. Judging effects based solely on text descriptions will almost inevitably miss defects.
5. Viewpoints should be embedded in interactions. If the site intends to express “creation is subtraction,” the best way is not to write an explanatory text, but to let users personally experience that “what is removed cannot be added back” through operation.

9. Conclusion

Using LLMs for 3D interactive narrative development is essentially splicing together two capabilities: “creative generation” and “engineering delivery.” The former relies on the model’s open-ended judgment, while the latter relies on clear constraints, verifiable阶段性 goals, and solid asset and performance handling. As long as the phases of requirement abstraction, environment assessment, state machine design, and acceptance testing are executed robustly, generative outputs can truly transition from “design documents” to “runnable, experienceable works.” For developers, this methodology is equally applicable to scenarios requiring both narrative and interaction, such as digital exhibitions, cultural science popularization, and product demonstrations.