LLM-Driven 3D Interactive Narrative: A Technical Implementation Path

1. From Requirements to Delivery: The Engineering Challenge of Autonomous LLM Creation

Large language models have become fairly mature in applications such as code generation and content creation. But when the task expands from producing a single text output to a complete autonomous chain—planning the work, researching materials, writing code, invoking 3D tools, and delivering a runnable website—the technical challenge shifts from a pure generation-quality problem to a systemic problem of environment awareness, tool orchestration, and multi-stage delivery.

A typical test scenario: ask the model to build an interactive website that explains to a general audience how a classic sculpture emerges from a block of raw stone. The requirement looks simple, but it spans narrative design, visual style definition, interaction approach, 3D model loading and rendering, and front-end engineering. The model must first assess what its runtime environment actually supports—whether it has file system access, whether it can execute code, whether it can make network requests to fetch assets—and then drive the project forward on its own until it delivers a runnable artifact.

In practice, models differ widely on this task. Some propose genuinely creative narrative angles but stall during implementation because of speed, cost, or toolchain limitations. Others default to a more conservative, exhibition-guide style of linear narrative: higher completion rates, but with compromises in interaction depth. This points to a core problem: a gap remains between the creative capability of large models and their engineering delivery capability, and it still needs to be closed.

2. The 3D Asset Pipeline: From Scan Data to Web Rendering

In a 3D interactive narrative project, asset acquisition and processing is the first technical hurdle. For classic sculptures, high-precision 3D scan data typically comes from public cultural heritage platforms, which generally offer STL or OBJ mesh data under open licenses for non-commercial or attributed use.

2.1 Data Format and Optimization

STL stores geometry as triangular faces: the structure is simple, but it carries no materials or texture coordinates. Loading a high-polygon STL model directly in the browser causes serious performance problems, so it needs to go through a processing pipeline like this:

// Pseudocode: model loading and simplification workflow
import { STLLoader } from 'three/examples/jsm/loaders/STLLoader';
import { SimplifyModifier } from 'three/examples/jsm/modifiers/SimplifyModifier';
const loader = new STLLoader();
loader.load('model.stl', (geometry) => {
  // Weld vertices to remove redundant ones
  geometry.mergeVertices();
  // Simplify the mesh to control the triangle count
  const modifier = new SimplifyModifier();
  const simplified = modifier.modify(
    geometry,
    Math.floor(geometry.attributes.position.count * 0.3)
  );
  // Recompute normals to ensure correct lighting
  simplified.computeVertexNormals();
  // Generate UVs (simple approach: planar or cylindrical projection)
  // ...
});

2.2 Progressive Loading and LOD

For an interactive website, first-load time directly affects user retention. A progressive loading strategy is recommended: serve a low-poly version first so users get an instant preview, then asynchronously fetch the high-poly version in the background and swap it in. You can also switch between levels of detail (LOD) automatically based on camera distance, keeping visual quality while controlling render load.

2.3 Materials and Lighting

The visual quality of sculptural 3D assets depends heavily on materials and lighting. On the web, this is usually handled with a physically based rendering (PBR) workflow, with a few things to watch:

• Marble requires an appropriate subsurface-scattering approximation to convey the warm, soft quality of the stone.
• In dark-room scenes, use spotlights or point lights to create dramatic contrast and emphasize the sculpture’s volume.
• Ambient occlusion (AO) noticeably improves how well fine details read; bake it or compute it in real time.

3. Interactive Narrative Architecture: From Linear Reading to Participatory Experience

Traditional digital exhibitions rely on linear narrative—users page through text, images, or video in a fixed order. The core value of interactive 3D narrative is letting users understand the creative process through their own actions. Two typical architectures:

Architecture	Representative approach	Strengths	Limitations
Immersive simulation	The user holds down a button to “chisel”; the stone peels away layer by layer and the sculpture gradually emerges	Strong sense of immersion; intuitively conveys the idea that “sculpting is subtraction”	High development complexity—requires real-time mesh deformation or keyframe-sequence animation
Slider-driven exhibition	A slider controls how much stone is removed, with staged reveals	Simple to implement; communicates information clearly	Limited interaction depth; weaker sense of participation

3.1 Implementing the Immersive Simulation Approach

For the “layer-by-layer removal” interaction, there are two viable approaches: precomputed keyframe sequences or real-time geometric clipping. The precomputed approach discretizes the process from a solid stone block to the finished sculpture into N keyframes and interpolates between them at runtime based on user input. The real-time approach dynamically “carves away” the stone with shaders or geometry operations—technically harder, but the interaction feels smoother.

A lightweight implementation idea: prepare two meshes—the complete stone block and the final sculpture—and use a vertex shader to drive vertex displacement with a “carving progress” parameter, producing a continuous transition from stone to sculpture.

3.2 Narrative Pacing and Copywriting

Interactive narrative is as much a content design problem as a technical one. Consider the “undo” button: in an ordinary product, undo is taken for granted. But in a carving context, clicking undo and seeing the message “marble has no undo key” turns a technical limitation into a metaphor for the irreversibility of sculpture. This kind of design requires narrative and interaction to be deeply coupled, not just features stacked on top of each other.

4. Automated Video Generation: Cross-Media Conversion from Website to Short Video

Once the website is delivered, the next task is converting it into a 60–90 second vertical short video for a broad audience. This stage involves:

4.1 Content Extraction and Restructuring

Websites and videos have different media characteristics. A website lets users explore freely; a video must deliver information along a fixed timeline. So you need to pull out the most striking visual moments and the core cognitive turning points from the site, then reorganize them into a “hook–development–climax–resolution” short-video structure.

4.2 The Technical Toolchain

Short-video generation typically requires the following capabilities:

• Screen recording and asset capture: record video clips of the key interactions on the website.
• Automated editing and transitions: stitch clips together automatically based on a script and add transition effects.
• Voiceover and subtitles: generate narration with text-to-speech and auto-align the subtitle timeline.
• Rendering and output: produce a 9:16 MP4 file while keeping an editable project file.

4.3 Quality Checks and Fixes

After automated generation, you must actually play the video back and fix common problems: audio and visuals out of sync, key information obscured, overly abrupt transitions, total duration exceeding the limit. This stage still requires substantial manual intervention and remains the weak point of the automated pipeline.

5. Key Findings from Engineering Practice

Several lessons from this work are worth passing on to developers:

First, assess the environment before designing the solution. When a task comes in, the model—or the developer—should first confirm: Is the file system accessible? Can code be executed? Can network requests be made to fetch assets? These constraints directly determine which technical routes are feasible.

Second, there is a “last mile” between creativity and delivery. Many models can propose excellent narrative ideas but fail to turn them into runnable code because of generation speed, context length, or tool-calling limits. Developers should budget enough time and compute for debugging and repair.

Third, cost and speed are the key variables for scaling. During prototyping, it is reasonable to use a high-capability model to explore the solution space. For batch generation or iterative optimization, though, you need to weigh model capability against invocation cost. In practice, a layered strategy—high-capability models for solution design and critical code generation, lighter models for repetitive code filling and testing—often delivers better cost efficiency.

Fourth, cross-media conversion needs dedicated optimization. Converting a website into a video is not simply “screen recording plus voiceover.” The content has to be reorganized around the target medium’s characteristics, and this stage deserves its own toolchain and QA process.

6. Summary and Outlook

LLM-driven 3D interactive narrative represents an important direction in AI-assisted creative development: the model is no longer just a content generator but an end-to-end executor spanning requirement understanding, environment assessment, technology selection, and engineering delivery. At the current state of the art, models can complete interactive websites of moderate complexity, but there is still clear room for improvement in real-time 3D rendering optimization, automated cross-media conversion, and the stability of long-chain tasks.

For developers, understanding the technical composition of this pipeline—and mastering the engineering methods for 3D asset processing, interactive architecture design, and automated video generation—makes it easier to put large model capabilities to productive use and turn ideas into deliverable, experiential products quickly. As model tool-calling capabilities strengthen and 3D-native generation technology matures, both the level of automation and the creative freedom in this field will continue to improve.