From Prompts to Interactive 3D: Evaluating and Practicing LLM Engineering Capabilities

1. A New Perspective on Evaluating LLMs: From “Answering” to “Delivering”

In the past, discussions about model strength were mostly limited to abstract metrics like benchmark scores and QA accuracy. However, a recent wave of community projects offers a more intuitive evaluation path: assessing whether a model can directly deliver a complete, tangible, and operable product.

We have seen reproductions of open-world style game blocks in web browsers, complete with streets, vehicles, and running characters, alongside toggleable graphics quality settings. Others have created 3D device demonstration pages where screen content responds in real-time to folding angles. Some have even broken down complex mechanical structures into over 2,000 parts for sequential interactive viewing, incorporating natural cloth simulation in wind fields.

The common value of these deliverables lies in compressing model capabilities into a perceptible interactive experience, allowing non-technical users to intuitively judge the model’s strength. For developers, the more practical question is: what capabilities does the pipeline from text to interactive 3D work rely on, and how can tasks be organized for speed and cost-efficiency? We will break this down below.

2. The Technical Essence of Three Typical Works

1. Web Game Prototypes: Testing Engineering Organization. Open-world style web demonstrations are essentially a model generating a complete front-end engineering project based on the browser’s 3D rendering capabilities in a single pass. This includes scene graph management, orbital camera controls, asset loading loops, input event dispatching, and graphics quality switching. Dozens of modules must collaborate with consistent interfaces. The difficulty lies not in any single snippet of code, but in cross-module consistency—this is the dividing line between “snippet completion” and “engineering organization.”

2. 3D Product Interaction Pages: Testing Toolchain Integration. A “model first, then web” combo is another typical pattern. The model first drives professional 3D modeling software to produce model assets, then imports these assets into the browser, binding fold animations and state linkages. Its significance lies in connecting modeling and interaction, upgrading product displays and creative proposals from static images to operable demos, significantly reducing communication costs.

3. Assemblies and Physical Simulations: Testing Domain Knowledge Depth. Cloth fluttering and mechanical structure disassembly require generated code to correctly implement rigid and soft body constraints, collision detection, and hierarchical assembly relationships in a physics engine. The volume of code and logical complexity are higher, making it a key scenario for distinguishing between “looks right” and “moves correctly.”

These three types of works point to a single conclusion: the role of LLMs is evolving from “code snippet completers” to “agents with engineering organization capabilities.”

3. Four Technical Foundations Supporting Creation

1. Engineering Level of Code Generation. Whether module division is reasonable, naming is consistent, state is centrally managed, and resource loading failures have fallbacks determines whether thousands of lines of code can be maintained long-term. Correctness within a hundred lines is just the entry level; structural consistency is the real threshold.

2. Agentic Tool Calling. Creative tasks typically advance in a multi-turn loop of “plan—execute—observe—correct.” The model calls external tools for results, self-repairs based on errors, and self-checks rendering effects in a preview environment. The stability of this loop directly determines whether the task can run unattended to completion.

3. 3D Graphics Knowledge Reserve. Scene graphs and transformation hierarchies, material and lighting models, Level of Detail (LOD), anti-aliasing, and performance budgets like framerate and draw calls will all be reflected in the quality of the generated code.

4. Physics and Simulation Knowledge. Rigid body dynamics, joint constraints, soft body (e.g., cloth) simulation, and integration stability with fixed time steps—these details determine whether the movement is natural and credible.

4. Practical Guide: Four Steps to Organize a 3D Interactive Creation

Step 1: Task Decomposition. Break “make a game demo” into five slices: scene, character control, camera, UI, and performance. Accept them slice by slice to avoid losing track of errors if a one-time generation goes out of control.

Step 2: Structured Prompts. A five-part structure is recommended:

[Role] Senior graphics programmer, familiar with browser 3D rendering and front-end engineering.
[Goal] Implement an interactive foldable device demo page.
[Feature Breakdown]
  1) Load 3D model assets.
  2) Orbital camera: drag to rotate, scroll wheel to zoom.
  3) Fold interaction: slider or double-click to switch between 0°~180°.
  4) State linkage: switch to widescreen UI when the unfold angle is greater than 90°.
[Constraints] Framerate no less than 30fps; provide graphics quality tiers; fallback prompts for asset loading failures.
[Acceptance] Provide three self-testable checklist items; self-test before delivery.

Step 3: Iteration and Checkpoints. Save to disk after completing each slice to allow rollback in case of regression. Write acceptance cases into the requirements so the model can self-test first, reducing manual rework.

Step 4: Minimal Skeleton First. Ask for a runnable empty scene first, then gradually overlay assets and interactions. An example skeleton is as follows:

// Pseudo-code: Minimal organizational structure for an interactive 3D scene
const engine = createEngine({ canvas: '#viewport' });
const scene  = engine.createScene();
const device = scene.loadModel('foldable-device');
device.bindInteraction('fold', {
  range: [0, 180],
  onProgress(angle) {
    device.setJoint('hinge', angle);                 // Hinge posture
    ui.switch(angle > 90 ? 'expanded' : 'compact');  // UI linkage
  }
});
engine.onTick(() => physics.step(1 / 60));           // Fixed time step
engine.start();

5. Cost Engineering: Estimation and Control under Token-Based Pricing

LLM services are generally priced by tokens (the unit of measurement for text and code processing). Agentic tasks follow a “multi-turn, rolling context” pattern: each turn must carry all previous conversations and code, causing the input-side cost to grow quadratically with the number of turns. The basic estimation formula is:

Single Task Cost ≈ Σ(Input Tokens per Turn × Input Unit Price) + Σ(Output Tokens per Turn × Output Unit Price)

Four methods to control costs:

• Sub-task Isolation: Split into smaller contexts to prevent infinite history expansion.
• Checkpoint Restart: After saving phased results, start a new session with a summary to continue.
• Prefix Caching: Utilize cache hits for stable system instructions and code skeletons to reduce repeated input costs.
• Debugging Sinking: Filter and refine error stacks with local tools before sending them to the model, reducing invalid turns.

From the magnitude of community reviews, agentic generation tasks lasting from a few hours to over ten hours can consume up to a million tokens, with costs ranging from a few dollars to tens of dollars. Switching to different models can cause the total cost of the same task to vary by an order of magnitude. Therefore, “cost per unit token” should be considered alongside “generation quality” as a second dimension for model selection.

6. Selection Evaluation Dimensions and Deployment

For selection, it is recommended to score based on five neutral dimensions: long-task code correctness, multi-turn tool calling stability, context length and cache support, unit cost, and compliance and data security.

At the deployment level, the interactive work itself is a static resource that can be distributed using object storage paired with a CDN. If you want to integrate the generation capability into your own business, you will need inference computing power and model services. You can select general categories such as GPU computing, model services, object storage, and content distribution on general cloud platforms like Baidu AI Cloud, and configure access control and auditing according to business requirements.

7. From Prototype to Production Engineering Checklist

A stunning prototype is only the first step. Moving to production requires supplementing: performance (mobile adaptation, rendering downgrade by device), loading (asset compression and lazy loading), robustness (exception fallbacks, degradation plans), maintainability (code reviews and module testing), and accessibility (keyboard reachability, contrast compliance). These engineering disciplines, combined with model capabilities, are what turn a “stunning demo” into a “reliable delivery.”

Conclusion

When the perspective of evaluating models shifts from scores to deliverables, the value of engineering methodology becomes prominent: decomposing capability foundations, organizing task slices, controlling token costs, and completing the production checklist. This methodology is not bound to any single model, yet it allows teams to achieve an interactive work with the lowest trial-and-error cost whenever a new generation of models emerges.