Evaluating LLM Engineering Capability in Practice: From Game Prototypes to 3D Interactive Applications

1. A Shift in Evaluation: From Benchmarks to Built Projects

Benchmark scores answer “does the model know it,” while engineering teams on the ground ask “can it actually build it.” In recent years, a pragmatic evaluation habit has emerged in the developer community: when a new large language model (LLM) is released, practitioners skip paper comparisons and go straight to completing a few visible, operable projects—a walkable open-world street scene, a foldable concept device that unfolds in the browser, or a mechanical assembly that can be disassembled into thousands of parts.

This “let the work speak” approach delivers value on three levels:

• Verifiable results: A project either runs or it doesn’t. Interaction smoothness is testable at a glance, sidestepping the score distortion caused by benchmark overfitting.
• Full-pipeline coverage: A single 3D interactive piece exercises code generation, rendering intuition, physical reasoning, and multi-turn bug fixing—far closer to production reality than isolated Q&A.
• Quantifiable cost: Running the same project with different approaches can produce token counts and total costs that differ by an order of magnitude, directly reflecting real-world economics.

2. Three Representative Practice Scenarios

2.1 Game Prototype: Open-World Scene Generation

A common stress test in the community is asking a model to generate an open-world street demo from scratch: roads, vehicles, pedestrians, lighting—all present, with keyboard-and-mouse character control. Key evaluation points include:

• Scene organization: Can the model sensibly split terrain, buildings, and vehicles into modules rather than dumping everything into one unmanageable file?
• Rendering trade-offs: Can it balance quality and frame rate on instruction—adjusting resolution scaling, shadow fidelity, and anti-aliasing settings?
• Game loop correctness: Can it implement a proper “input → update → render” main loop with consistent frame-to-frame state and stable time steps?

Community reports show that completing such a prototype typically consumes millions of tokens and costs tens of dollars, with human involvement ranging from a few hours to just over ten. For pre-production game research and gameplay validation, a “playable demo in hours” cadence dramatically shortens the pre-project trial cycle—prototypes that once required a small team working for days can now reach a first version in a single person’s single day.

2.2 3D Interactive Web Pages: Modeling Meets the Browser

A more advanced practice is the “modeling plus web deployment” pipeline: drive a professional 3D modeling tool via script to produce assets, import those assets into a web page, render them through standard web 3D APIs, and bind interaction logic. A typical example is a concept foldable-screen device web page—users drag to unfold or fold the body, and on-screen content updates with the device’s form factor.

The core challenge here is cross-tool orchestration:

Requirement description → Modeling script generation → Asset export → Browser loading → Event binding → Form-factor-aware rendering

Every step is error-prone: mismatched coordinate conventions, flipped normals, missing texture paths, misaligned animation timelines. Whether a model can autonomously locate and fix such cross-stage issues over multiple iterations is the dividing line between “can write code snippets” and “can finish engineering work.” Direct applications include product 3D showcases, creative proposals, and marketing pages—work that previously took modelers and front-end engineers days can now reach a first cut in hours, with humans polishing the details afterward. Community data suggests combined modeling-and-web-deployment call time of roughly one to two hours, with costs typically around ten dollars, varying by iteration count and context length.

2.3 Physics Simulation and Assembly Disassembly: The Litmus Test for Dynamic Realism

A static render that looks good is just a passing grade; “does it move naturally” better distinguishes capability tiers:

• Soft-body simulation: Cloth fluttering with wind and character movement tests the model’s grasp of stiffness, damping, and collision parameters. Slightly wrong parameters make cloth look like a steel plate or like jelly—a classic “looks right, is actually wrong” domain that demands frame-by-frame human review.
• Assembly disassembly: Decomposing a mechanical hand with over two thousand parts while preserving hierarchical structure, assembly relationships, and naming conventions. Every order-of-magnitude increase in part count significantly raises the probability of naming conflicts, hierarchy chaos, and duplicate references.

Community examples show ten-minute-scale completions for such tasks. Maintaining structural consistency across thousands of parts at that speed indicates the generation approach has reached an engineering-grade level of long-range code organization, with direct transfer value to manufacturing disassembly guides, maintenance training, and digital-twin displays.

3. The Generative Development Workflow

Abstracting from the practices above yields a general workflow:

1. Task decomposition: Split a large goal into a sequenced set of stages—“scene skeleton → core assets → interaction logic → performance tuning”—each with its own acceptance criteria.
2. Context management: Inject only the necessary history at each stage; use phase summaries instead of the full history to avoid irrelevant content crowding the input window.
3. Incremental validation: Run each completed stage in a browser immediately, feeding full error stacks and observed behavior back to the model.
4. Version control: Store generated artifacts in a regular code repository for rollback capability and for side-by-side comparison of outputs across models and prompting strategies.

A typical skeleton for an interactive 3D scene main loop looks like this:

// Main loop: input → simulation → render
function tick(now) {
  const dt = Math.min((now - lastTime) / 1000, 0.05); // Cap step size to prevent simulation jumps after tab switch
  lastTime = now;
  controls.update(dt);      // Map keyboard/mouse input to camera / character
  physics.step(dt);         // Advance cloth, rigid-body, etc. simulations
  scene.traverse(updateAnimation);
  renderer.render(scene, camera);
  requestAnimationFrame(tick);
}

Code quality here hinges heavily on understanding timing constraints in the main loop—for example, decoupling physics step size from rendering frame rate and capping delta time to prevent objects from “teleporting” after a tab switch. These details are reliable signals of a model’s engineering maturity.

4. Cost Engineering: Token Accounting and Budget Estimation

4.1 What Is a Token?

A token is the basic unit LLM services use to measure text and code. Roughly, 0.75 English words equal one token; a Chinese character typically maps to one or two tokens; indentation and punctuation in code count too. Input and output are generally priced separately, with output often costing several times more per token than input. In code-generation scenarios, output costs can be substantial, so budgeting on conversation turns alone is misleading.

4.2 How to Estimate Costs

A practical estimation model:

Total cost ≈ Input tokens × Input price + Output tokens × Output price + Add-on service fees

Three key factors drive total cost:

• Context length: More interaction rounds and longer histories make input costs grow linearly or faster.
• First-pass success rate: The share of generations that run on the first try directly determines how many fix cycles—and how much extra spend—you incur.
• Cache utilization: Most model service platforms support prefix caching; hits dramatically reduce billing. Engineering-wise, keep the prompt prefix stable and put variable content toward the end.

A walkthrough for a hypothetical mid-size project: 120 interaction rounds, averaging 25,000 input tokens and 1,200 output tokens per round, yields roughly 3 million input tokens and 144,000 output tokens. At $3 per million input tokens and $15 per million output tokens, input runs about $9 and output about $2.20, totaling roughly $11; with retries and extra calls, the real bill typically lands in the teens to low tens of dollars—consistent with community reports of “millions of tokens, tens of dollars per prototype.” Notably, when per-token prices are similar across providers, cost differences between teams come mainly from iteration efficiency. Wasted rounds bloat the bill; prompt engineering literally saves money.

5. Evaluation Methodology: Building Your Own Benchmark Suite

Rather than debating which model is stronger, freeze a set of “benchmark projects” for your team and run them through a unified process for reproducible conclusions:

Dimension	What to Observe	How to Record
Runnability	Does the first generation run?	Pass on first try / Pass after N fix rounds / Fail
Interaction naturalness	Input lag, animation smoothness	Frame-rate data + 5-point subjective score
Structural quality	Module split, naming, maintainability	Manual code review
Cost efficiency	Token consumption, total cost, wall-clock time	Measured logs
Long-range consistency	Do earlier features degrade after many rounds?	Regression checklist

Execution essentials:

1. Fix prompts and acceptance criteria; only swap the model under test to ensure horizontal comparability.
2. Log token counts, costs, rounds, and time for every run; build an internal cost baseline over time.
3. Re-run benchmarks periodically to track capability shifts across model versions—don’t decide on stale impressions.

For teams that will rely on models in production long-term, consider wiring benchmark projects into a CI pipeline: whenever a model or prompt strategy updates, auto-rerun and emit a comparison report, eliminating subjective drift from “just try it manually.” On the infrastructure side, cloud platforms such as Baidu AI Cloud that offer model services, container runtimes, logging, and monitoring can be used to containerize evaluation tasks and centralize consumption data for auditable selection decisions.

6. Best Practices and Pitfalls

1. Prefer small tasks over large ones: Asking for “the whole game” at once is riskier than staged generation with independent checkpoints—failures stay contained.
2. Front-load acceptance criteria: State frame-rate targets, supported interactions, directory structure, and error-handling conventions in the prompt to reduce back-and-forth.
3. Give differentiated feedback: When reporting issues, supply the full stack trace, reproduction steps, and expected behavior—far more useful than “this has a bug.”
4. Tame context bloat: Summarize completed work at each stage and use summaries instead of concatenating the full history—balance memory against cost.
5. Keep humans in the loop: Physics parameters, numerical stability, and performance metrics are easy to “look right” but get wrong; review them manually and codify automated regression tests where possible.

7. Conclusion

From open-world street scenes and foldable-device web pages to mechanical hands disassembled into thousands of parts, “let the work speak” is becoming the pragmatic path to evaluating LLM engineering capability. For development teams, the assets worth accumulating are not fleeting impressions of any single model, but a reproducible benchmark suite and a continuously updated cost ledger—tools that let you answer, with every new model release, the most practical question of all: can it do the same job faster, cheaper, and more reliably?