From Code Generation to Runnable Artifacts: A Technical Review of LLMs’ 3D and Interactive Capabilities

Recently, following the release of a new version of a domestic large language model (LLM), social platforms have seen a surge of user-generated works: open-world-style city scene demos, foldable mobile device models that actually unfold in a browser, natural cloth simulations, and mechanical structures that can be exploded into over 2,000 individual parts. What these works share is that they are no longer just “text that looks like code”—they are complete, openable, operable, and observable artifacts. This shift signifies that the capability boundary of LLMs is extending from “language and code generation” to “interactive 3D content production.” This article extracts key technical points from these public cases to discuss the underlying capability composition, cost structure, and engineering implementation methods.

I. The Core Shift: From Text Output to Runnable Artifacts

Traditional code generation models are typically evaluated based on syntax correctness, function completeness, and unit test pass rates. However, the evaluation criteria for this recent wave of showcased works have moved up a level: whether the artifact runs, whether the interaction is natural, and whether the visual result is credible.

Take the widely circulated open-world scene demo as an example. Through natural language description alone, users obtained a real-time demo featuring street layouts, moving vehicles, and running character animations. Such a task involves at least four concurrent capabilities:

1. Structured Scene Description: Breaking down vague concepts like “city block” into computable elements such as road grids, building masses, lighting directions, and camera paths.
2. Accurate Graphics API Invocation: Correctly organizing the rendering pipeline, material systems, and animation loops.
3. Spatial Reasoning: Ensuring that object proportions, occlusion relationships, and motion trajectories are self-consistent in 3D space.
4. Iterative Correction: Making local adjustments based on follow-up prompts rather than rewriting the entire codebase when the generated result does not meet expectations.

Notably, users mentioned improving visual fidelity by instructing the model to “enable a certain super-resolution rendering option.” This indicates that the model can not only generate initial code but also understand and tap into the post-processing stages of the rendering pipeline, demonstrating a degree of “graphics context awareness.” For developers, this means the model can act as a collaborator that understands graphics terminology, rather than just a code typewriter.

II. Technical Breakdown of Three Typical Tasks

1. Real-time 3D Scenes: State Management and Performance Constraints

The core difficulty of an open-world demo is not “drawing houses and cars,” but rather continuous state management. An interactive scene needs to maintain: player/camera position, object velocity vectors, collision detection, animation timelines, and render frame loops. The model must generate a complete program structure with a main loop, rather than a static frame.

From an engineering perspective, the value of such artifacts is concentrated in the prototyping phase:

• Rapid Atmosphere Validation: Validating camera language and controls using placeholder geometry before committing actual art resources.
• Lowering the Graphics Programming Barrier: Developers can describe desired effects in natural language, letting the model handle the initial setup of shader parameters and scene graphs.
• Iterability: Since the output exists as code, teams can manually optimize it further rather than being locked into a black-box tool.

It is worth noting that these generated artifacts are typically still rough in terms of performance optimization, memory management, and resource loading strategies, making them a starting point rather than a final state. Production environments still require professional graphics engineers.

2. Interactive Product Showcases: Bridging Modeling and the Web

The foldable device demo perfectly illustrates the concept of “pipeline bridging.” The user’s request essentially consisted of two tasks: first, constructing a model with joint structures in a 3D modeling environment, and second, exporting it into a browser-operable format where the screen content dynamically links to the folding action.

This pipeline involves several key technical aspects:

• Hierarchy and Skeletal/Joint Structure: Folding actions require correct parent-child transformation relationships, and rotation pivots must be accurate.
• Interaction Event Binding: The folding progress needs to drive changes in the screen texture’s display area, creating a “shape-content” coupling.
• Lightweight Export: Web-side rendering is sensitive to polygon counts and texture sizes, requiring trade-offs between visual quality and loading performance.

Direct application scenarios for this capability include product showcase pages, creative proposals, and interactive manuals. Compared to the traditional sequential workflow of “modeler models + frontend engineer writes interaction,” natural-language-driven generation compresses both steps into a single conversation, significantly shortening the cycle from idea to demonstrable artifact. For SMBs, this means the marginal cost of product visualization prototypes drops significantly.

3. Physics Simulation Tasks: The Source of Naturalness

The cloth simulation and mechanical structure explosion cases point to another layer of capability: modeling physical laws and assembly logic.

The “naturalness” of cloth simulation usually stems from the correct implementation of mass-spring models or Position-Based Dynamics (PBD), including the rational setting of gravity, constraint iteration counts, and damping coefficients. The model needs to understand the numerical meaning behind the word “flutter” rather than simply stacking vertex animations.

In the mechanical explosion case, the artifact needed to generate a structure with over 2,000 parts in under ten minutes and support an explosion animation. This requires the model to possess:

• Assembly Hierarchy Reasoning: The containment relationships and disassembly sequence between parts.
• Batch Geometry Generation: Generating massive structures procedurally rather than via manual modeling.
• Animation Orchestration: Planning displacement paths and timing for each part.

Such capabilities hold potential value in industrial showcases, educational demos, and maintenance manuals. However, it must also be pointed out that the engineering accuracy (dimensional tolerances, material properties, assembly feasibility) of these generated structures cannot yet replace professional CAD workflows. Its positioning should be an “expression and communication tool,” not a “manufacturing basis.”

III. Cost Structure: An Overlooked Key Variable

A set of numbers repeatedly appearing in public cases deserves separate discussion: costs in the tens of dollars, consumption of millions of tokens, and generation times measured in hours. Looking at these metrics together yields several practically useful conclusions for developers.

First, the token-based billing model makes “trial-and-error costs” budgetable. In traditional outsourcing or manual development, a directional mistake could cost days of labor; in a generative workflow, the cost of starting over is a calculable incremental fee. This makes “parallel exploration of multiple solutions” economically viable.

Second, cost correlates non-linearly with task complexity. Generating a simple short clip might cost single-digit dollars, while a fully interactive 3D scene increases the cost by an order of magnitude. When planning tasks, developers should estimate expenses based on “whether continuous running state is required” and “whether multi-round interactive corrections are needed,” rather than estimating by lines of code.

Third, be cautious with horizontal comparisons. Community claims that “the same task costs several times more on other models” are heavily influenced by prompt quality, retry counts, and task decomposition granularity, making them unsuitable as the sole basis for selection. A safer approach is to build a small benchmark for your typical tasks and record two metrics: the success rate of generating a runnable artifact on the first try and the average cost per successful attempt.

IV. Practical Recommendations for Developers

Based on the observations above, here are a few actionable engineering recommendations.

1. Task Decomposition Over Prompt Sculpting

The one-shot success rate for complex 3D artifacts is limited. A more effective strategy is a layered approach:

`text Layer 1: Generate the scene skeleton (camera, lighting, ground) Layer 2: Add main objects and layout Layer 3: Add interaction and animation logic Layer 4: Locally refine materials and parameters

Generate a runnable minimal version at each layer, then incrementally build upon it. This reduces the complexity of single generations and makes error localization easier.

2. Establish a “Runnability” Acceptance Checklist

Establish minimum acceptance standards for generated artifacts, such as:

• Whether it can launch without errors in the target runtime environment;
• Whether the main loop is stable and there is no obvious memory growth;
• Whether interaction events trigger as expected;
• Whether resource references use relative paths and are migratable.

Solidifying this checklist into team standards prevents the rework of “looks good but doesn’t work.”

3. Retain Manual Optimization Interfaces

Generated code should be treated as a first draft. Teams should retain clear configuration options and comment anchors at key positions to facilitate handoff and optimization by graphics or frontend engineers later. Avoid delivering generated results directly as unmodifiable black boxes.

4. Focus on Artifact Portability

3D and interactive artifacts have strong dependencies on their runtime environments. During model selection and workflow design, prioritize solutions based on universal web standards (e.g., WebGL/WebGPU interfaces, standard 3D file formats) to avoid vendor lock-in. This ensures artifacts can be reused across custom pages, embedded browsers, or demo environments.

V. Conclusion

The most noteworthy aspect of this recent wave of demos is not the improvement of a single metric, but the change in capability modality: the object output by the model has shifted from “text” to “runnable, interactive, observable artifacts.” For developers, this means the prototyping toolchain is being redefined—shifting partially from “writing code to implement an idea” to “describing an idea and receiving a first draft.”

At the same time, it is vital to stay grounded: generated artifacts still have clear boundaries regarding performance, precision, and engineering standards. Their optimal positioning is to accelerate exploration and communication, not to replace professional engineering implementation. By calculating the cost structure clearly, establishing acceptance standards, and leaving room for manual optimization, teams can truly transform these capabilities into productivity, rather than just flashy demo videos.