Multimodal LLM-Driven 3D Interactive Content Generation in Practice

1. From an Open-Ended Prompt to a Deliverable Work

“Build a Chinese-language interactive website that helps ordinary people understand how a classic sculpture emerges from a block of stone.” This is a typical open-ended creative task: the goal is clear, but the path is entirely unconstrained. Narrative angle, visual style, interaction model, and technical approach are all left to the model to decide. Such tasks best test the real engineering capability of multimodal large models—the model must not only “have ideas” but also “carry them through to delivery.”

In actual testing, different models produced markedly different proposals. One approach leaned toward an immersive dark-room experience, where viewers repeatedly “chisel” away at the stone to peel it back layer by layer, gradually revealing the sculpture, with metaphorical copy such as “marble has no undo key” embedded in the interaction. Another approach leaned toward an exhibition-catalog style, presenting scenes in stages against a paper-colored background and using a slider to control the stone’s progressive reduction from block to finished piece. Both approaches are viable, but they differ noticeably in engineering completeness, responsiveness, and cost.

This article does not review the performance of any specific product. Instead, it abstracts this class of tasks into a general technical problem: how to use multimodal large models to build runnable, experienceable 3D interactive narrative content, and it presents a reusable engineering method.

2. Task Decomposition: Turning “Creation” into Verifiable Milestones

The biggest risk of open-ended prompts is that they cannot be accepted or verified. The first step of engineering is to break the task into verifiable stages:

1. Environment capability assessment: Does the current runtime support 3D rendering, model loading, and asset hosting? What are the browser-side WebGL/WebGPU capabilities? Are external resource requests allowed?
2. Asset acquisition and compliance: Are high-precision 3D scan assets available? Does the license permit redistribution? What attribution information must be retained?
3. Narrative structure design: How many scenes? What is the core interaction action? What should viewers understand at each stage?
4. Technical approach determination: 3D model format, rendering pipeline, interaction event binding, performance degradation strategy.
5. Deliverable definition: An accessible page, a playable video, an editable project, and an asset source list.

Each stage should have clear pass criteria. For example, the criterion for the asset stage is “the model loads, the polygon count is acceptable for target devices, and licensing information is complete.”

3. Acquiring and Processing 3D Assets

High-quality 3D scan assets are the foundation of this kind of content. Public cultural heritage scanning projects typically provide models in STL or OBJ format, usually under Creative Commons-type licenses that permit use and redistribution with attribution.

Several key points require engineering attention:

• Polygon count and file size: Raw scan models often have extremely high polygon counts, and loading them directly leads to excessive first-screen load times. Mesh simplification is needed to reduce the polygon count to a level browsers can render smoothly while preserving contour features.
• Format conversion: STL describes only geometry, without materials or UVs. If richer rendering effects are needed, convert to glTF/GLB for easier loading on the web.
• Coordinate and scale normalization: Models from different sources use inconsistent coordinate systems, so orientation and scale must be unified during preprocessing to avoid repeated debugging in the scene.
• Embedding license information: Writing source, author, and license into page metadata or an about page is both a compliance requirement and part of content credibility.

A typical preprocessing pipeline can be abstracted as:

Raw scan (STL/OBJ)
  → Mesh cleanup (denoising, hole filling)
  → Simplification (target polygon range)
  → Retopology / normal recalculation
  → Export glTF/GLB
  → Compression (Draco / Meshopt)
  → Upload to object storage + CDN distribution

4. Web-Side 3D Rendering and Interaction Design

The mainstream approach for browser-side 3D rendering is a WebGL-based rendering library paired with a glTF loader. Core interaction patterns include:

• Progressive reveal: Simulating the process of “stripping away from stone” through clipping planes, transparency gradients, or geometry masks. In implementation, the clipping plane position parameter can drive the effect, with animation curves controlling pacing.
• Slider-driven: Binding clipping parameters or the model’s visible ratio to a slider lets viewers drag freely, suiting an “exhibition catalog” style of narrative.
• Button-driven: Each click advances one stage, suiting narratives with clear chapters and making it easy to embed copy feedback.
• Camera control: Restricting the range of orbit controls prevents viewers from rotating to meaningless angles while ensuring key viewpoints remain reachable.

For performance, degradation strategies must be prepared: when a device lacks advanced features or frame rate is insufficient, reduce model precision, disable shadows, and cut post-processing. Interaction events should be throttled to avoid high-frequency triggers blocking the main thread.

5. Cost and Quality Control for Multimodal Content Generation

Beyond the 3D website, similar tasks often require a companion short video: 9:16 vertical, 60–90 seconds, aimed at a general audience, delivered as a playable MP4 plus an editable project. This involves text-to-video or multimodal generation capabilities.

In actual engineering, the following require attention:

• Cost and latency: High-quality video generation has high unit cost and long turnaround. The number of shots and maximum duration should be fixed at the scripting stage to avoid repeated regeneration.
• Consistency: If the video reuses website assets (models, color schemes, copy), visual style must remain unified. It is advisable to export key frames or rendered clips from the website first and use them as reference input for video generation.
• Editability: Delivering an editable project means retaining layered assets, timeline, and audio tracks for later modification. Purely generative output is often hard to re-edit, so a compositing step must be reserved in the process design.
• Quality checks: Finished videos need manual or automated checks for obvious issues such as tearing, misaligned subtitles, audio-video desynchronization, and missing asset attribution.

6. A Reusable Engineering Methodology

Distilling the above practices into a methodology yields five points:

1. Assess before acting: Clarify the runtime environment, available assets, and licensing boundaries before deciding on a technical approach.
2. Milestone-driven: Break open-ended creation into acceptable stages, each with clear pass criteria.
3. Assets first: The bottleneck in 3D content is usually asset processing; complete simplification, format conversion, and distribution preparation early.
4. Interaction serves narrative: Interaction actions should serve cognitive goals, not showmanship. Each interaction should correspond to an information point viewers can understand.
5. Delivery is verification: Only when it is runnable, experienceable, editable, and has complete asset sources is it truly delivered.

7. Conclusion

Multimodal large models have moved “generate a 3D interactive website from one sentence” from demo to practice, but the key to engineering deployment still lies in task decomposition, asset processing, rendering optimization, and cost control. Only by converting open-ended creation into a verifiable engineering process can a stable path be established between creativity and delivery. For enterprise users, this methodology also applies to digital showrooms, cultural heritage digitization, and interactive marketing: define acceptable milestones first, then let the model exercise creativity within those boundaries.