Practical LLM Code Generation: From Prompts to 3D Interactive Applications

1. Observation: AI-Generated Works Evolve from “Viewable” to “Playable”

Recently, the developer community has seen a wave of representative AI-generated projects, which can be categorized into three main types:

• Open-world style game prototypes: Complete street scenes featuring roads, buildings, moving vehicles, and walking pedestrians. Some creators even attempted to apply common graphics enhancement techniques like super-resolution and frame generation to improve visual quality.
• Interactive foldable device models: Constructed using 3D modeling concepts and then ported to the web. Users can drag to fold or unfold the device, with the screen content dynamically updating based on the folding angle.
• Mechanical structure disassembly demonstrations: A mechanical hand broken down into over 2,300 individually viewable parts, complete with physical effects like cloth simulation, resulting in a highly polished visual experience.

The commonality among these cases is that the outputs are no longer static pages or simple animations, but 3D applications with complete state management and interaction logic. Based on publicly shared production data, complex game prototypes take about ten hours of iteration, consume millions of tokens, and cost in the tens of dollars. Meanwhile, a promotional short with 3D graphics can be generated from prompt to final render in just a few minutes for single-digit dollars. Low cost and high completion are the fundamental reasons these works sparked widespread discussion.

2. Technical Breakdown: How LLMs Can Write 3D Applications

1. The Web 3D Tech Stack is Highly Standardized

Modern browsers provide WebGL/WebGPU rendering interfaces, complemented by mature scene graph and rendering encapsulation solutions. This makes the code structure of 3D applications highly fixed: scene, camera, renderer, lighting, materials, and animation loop. The skeleton remains uniform; what changes is the scene content and interaction logic. This standardization perfectly aligns with LLM strengths—there is massive amounts of similar code in the training data, allowing models to reliably reproduce the engineering skeleton and focus “creativity” on the content layer.

2. Procedural Generation Naturally Fits “Code as Asset”

Content like open-world street scenes does not rely on fine art assets. Procedural generation can produce buildings, road networks, and traffic flows using rules and random numbers. For a model, writing “code that generates assets” is far more efficient than directly describing the assets themselves. Combining the two exponentially amplifies the information density of a single generation.

3. Long Context Supports Multi-file Engineering

Complex interactive applications inevitably split into multiple modules: scene organization, entity logic, physics simulation, and UI panels. Longer context windows and multi-file editing capabilities allow the model to maintain a global architectural perspective within a single session, handling cross-file references and consistency modifications.

4. Math and Physics Knowledge Can Be Directly Coded

Cloth simulation relies on mass-spring models or Position Based Dynamics (PBD) constraints. Pedestrian movement depends on navigation meshes and state machines. Folding mechanisms rely on hinge rotational degree of freedom constraints. These algorithms are ubiquitous in textbooks and open-source implementations, allowing the model to translate theory directly into runnable code.

3. Practical Workflow: From a Single Prompt to a Runnable Application

1. Break Large Goals into Verifiable Small Tasks

Instructions like “build a city driving prototype” are too vague and often result in seemingly complete but non-runnable code. A more effective approach is phased execution: first set up the rendering skeleton and run an empty scene, then incrementally add road network generation, vehicle movement, camera controls, and UI panels. Each step can be independently verified.

2. Explicitly Declare Engineering Constraints in Prompts

Implement an interactive city street scene prototype using web 3D rendering technology. Requirements:
1. Scene: 12x12 block grid, including roads, buildings, streetlights;
2. Dynamics: Vehicles driving in loops along the road network, pedestrians walking randomly along sidewalks;
3. Interaction: Mouse drag to rotate, scroll wheel to zoom, click buildings to pop up an info panel;
4. Engineering constraints: Split code into scene / traffic / ui modules;
5. Performance constraints: Use instanced rendering to control draw calls, target 60 FPS.

3. Organize Real-time Logic with a Main Loop

// Pseudo-code: Adaptable to any mainstream web 3D rendering solution
const scene = createScene();
const camera = createPerspectiveCamera({ fov: 60 });
const renderer = createRenderer(canvas, { antialias: true });
// Procedurally generate city blocks
const city = buildCity({ blocks: 12, roadWidth: 8, heightRange: [10, 60] });
scene.add(city);
function tick(t) {
  updateTraffic(city, t);      // Vehicles moving along the road network
  updatePedestrians(city, t);  // Pedestrian animation state machine
  camera.follow(playerCar);    // Camera follow
  renderer.render(scene, camera);
  requestAnimationFrame(tick);
}

4. Feed Execution Results Back to the Model

The key to iterative debugging is giving the model real feedback. Paste error stacks directly back for fixes. If performance falls short, provide frame rate and draw call statistics. If there are visual deviations, describe the phenomenon or attach a screenshot. Fix requests with specific feedback have a much higher success rate than a vague “rewrite it.”

4. Cost Perspective: Engineering Management of Token Consumption

Tokens are the unit of measure for how models process text and code. The total consumption of a task is determined by both the input context and the output length. There are three core strategies for controlling costs:

• Incremental modifications over full rewrites: Ask the model to only change the target function or module, avoiding repeated output of unchanged code.
• Streamline context: Prune irrelevant historical conversations and verbose logs promptly, keeping only the file snippets necessary for the current task.
• Module reuse: Consolidate stable utility functions into a local library, using references instead of regenerating them.

Based on publicly shared community figures, small to medium 3D interactive projects typically cost anywhere from a few dollars to tens of dollars, primarily depending on the number of iterations and the context size per round. Treating cost as an engineering metric to manage, rather than a surprising bill after the fact, is a prerequisite for leveraging these capabilities effectively.

5. Typical Use Cases

• Game and gameplay prototypes: Validate core gameplay mechanics at extremely low cost before investing art and engine teams.
• Product 3D showcases: Web-based interactive demos for foldable devices, furniture, and industrial equipment, replacing some physical photography and video production.
• Educational and disassembly demonstrations: Mechanical structure exploded views and layered structure displays, turning abstract objects into operable 3D content.
• Data visualization: Rapid construction of 3D geographic information and digital twin scenes for industrial parks.

6. From Local Demo to Online Service

Turning demo code into an accessible service requires deployment. The general approach is: host static resources (page scripts, model files) on object storage, accelerated by a CDN; deploy components involving real-time data or server-side simulation on container platforms or GPU cloud servers; and build observability through logging services and monitoring alerts. Taking Baidu AI Cloud as an example, general product categories like object storage, CDN, container services, and cloud monitoring can be combined on demand, allowing developers to go live without building their own operations and maintenance infrastructure.

7. Stay Grounded: The Distance Between Demos and Production

Beyond the excitement, it is important to rationally view the gap between these works and production-grade applications:

1. Lack of performance depth: Demo works rarely handle deep optimizations like draw call batching, LOD (Level of Detail) grading, or texture atlasing. Frame rates will drop rapidly as scene scale increases.
2. Limited physics accuracy: Cloth and collisions are mostly visual approximations and cannot be directly used for engineering simulations.
3. Questionable maintainability: Generated code lacks unified design conventions. Long-term iteration requires manual refactoring and adding tests.
4. Compliance risks: The style and assets of generated content need to have their copyright boundaries manually verified.

The conclusion is clear: treating LLMs as “high-efficiency prototype engines” rather than “silver bullets that replace engineers” is the most pragmatic positioning at present.

8. Conclusion

From text to images, and now to directly runnable 3D interactive applications, the output of generative AI is continuously becoming “heavier.” For developers, the real opportunity lies not in watching demos, but in mastering the complete methodology of “requirement breakdown — prompt design — feedback iteration — cost control — cloud deployment,” consolidating the model’s capabilities into reusable engineering workflows. The next batch of amazing works will likely come from you putting things into practice after reading this article.