Evaluating LLM Coding Ability Through 3D Interactive Deliverables

1. From “Higher Benchmark Scores” to “Let the Work Speak”

Coding models are iterating faster than ever, with new versions released at a steady clip and leaderboard scores rising accordingly. But for working developers, the more practical question is: when a score goes up by one point, what does that actually mean for my project?

Recently, a wave of hands-on build reports has emerged around a new batch of coding models in overseas technical communities. Some developers built open-world-style city scene demos, others turned a dual-screen foldable device into a web page that unfolds and folds in the browser, and still others recreated a mechanical structure that disassembles into more than two thousand parts. What these works share is that you can look at them and interact with them directly.

This suggests an evaluation approach more convincing than benchmark scores: hands-on evaluation with deliverable works as the acceptance criterion. There are three reasons:

1. Comprehensiveness: A working interactive piece simultaneously tests code generation, graphics knowledge, state management, and performance awareness. A weakness in any one area shows up on the final screen.
2. Reproducibility: Anyone can reproduce the same result using the same natural-language requirements, with cost and time clearly measurable.
3. Closeness to real needs: Game prototypes, product demo pages, and interactive proposals are engineering scenarios that a large number of teams genuinely face.

2. Four Typical Deliverables and What Each One Tests

2.1 Open-World-Style City Scene

One community user reported spending about 10 hours and roughly 4 million tokens to build a city scene demo with streets, moving vehicles, and running pedestrians, at a total cost of about $52. The author also tried enabling render-level supersampling and image-quality enhancement options and showed the difference before and after the adjustment.

The tech stack behind this kind of work is not simple:

• Scene graph organization: A city-scale scene means hundreds or thousands of renderable nodes, requiring a sound hierarchy and culling strategy—otherwise the frame rate collapses immediately.
• Procedural generation: Street layouts and building placement are usually not hand-placed; the model needs to write procedural generation logic.
• Vehicle and character control: This involves input mapping, kinematics, camera follow, and other subsystems working together.
• Rendering optimization awareness: Whether the model proactively suggests quality tiers, on-demand rendering, and similar optimizations directly reflects its engineering maturity.

Notably, the same user estimated that doing the same work with some comparable commercial models could cost nearly an order of magnitude more. This claim is a personal estimate and not necessarily rigorous, but it points to an often-overlooked selection dimension: token consumption for the same task can vary enormously across models, and it is worth testing with your own tasks.

2.2 Interactive Web Page for a Foldable Device

Another widely discussed case is a concept demo of a dual-screen foldable device. The author had the model first produce the device model in mainstream 3D modeling software, then wired the model into a web page to enable interactive unfolding and folding, with screen content switching in real time as the form changes. The whole workflow took about 1.5 hours and cost about $12.5.

The value of this case lies in connecting two pipelines that are usually separate: modeling and front-end interaction. Producing a 3D asset first and then making it move and respond to input in the browser means the model must understand both the modeling software’s scripting system and the web-side rendering and event mechanisms. For work like product showcases and creative proposals, this pipeline upgrades a “static render” into a “playable prototype.”

2.3 A Launch Short Film Produced in Five Minutes

Another user had the model generate a 3D animated short film directly: about 5 minutes from requirement to finished piece, at a cost of about $1.1. The approach was to have the model write animation code based on browser-based 3D rendering—essentially “shooting” a short film with code rather than through a traditional editing workflow.

This shows that the pipeline from text to motion visuals is already short enough. It is more than sufficient for lightweight scenarios like internal proposals and release announcements, and the whole thing is parameterizable, so the cost of revising a version of the copy is almost zero.

2.4 Cloth Simulation and a Two-Thousand-Part Mechanical Disassembly

Dynamic behavior is where 3D work best distinguishes strong capability from weak. One user tested cloth flutter simulation and a mechanical structure that disassembles into 2,317 parts, both completed within 10 minutes.

• Cloth simulation tests the tuning of physical parameters: stiffness, damping, wind fields. Improper parameters cause clipping or jitter. Whether the model can adjust parameters autonomously based on visual feedback is an important sign of agentic capability.
• Part disassembly (exploded view) tests large-scale node management: more than two thousand parts means the scene graph depth, naming conventions, and animation choreography all have to hold up. A little disorder produces misplaced and flickering parts.

3. Decomposing the Capability Stack Behind the Deliverables

Beneath these flashy demos, the supporting capabilities can be grouped into four layers:

Layer 1: Long-session consistency. Consuming millions of tokens means the entire production process spans a large number of turns. The model needs to still remember the original requirement constraints and the technical choices already made after the context has ballooned, without “changing its mind mid-way” drift.

Layer 2: Cross-domain knowledge fusion. Graphics terminology, physical parameters, front-end engineering conventions, and modeling software scripting systems—this knowledge must be correctly combined and used within a single session.

Layer 3: Agentic workflow. High-quality delivery almost inevitably depends on a generate–run–observe errors–fix loop. Whether the model can read logs on its own, locate problems, and make minimal changes determines whether ten hours produce a pile of runnable fragments or a coherent whole.

Layer 4: Interaction and motion detail. The easing curves of a folding animation, the UI state switching after unfolding, the sense of rhythm in camera movement—these “feel” aspects are the hardest to fake and the most easily perceived by end users.

The conceptual skeleton below (independent of any specific framework) roughly describes the core structure of this kind of interactive application and can serve as a reference for checking code quality during acceptance:

// Conceptual example: a generic skeleton for a 3D interactive app
class InteractiveApp {
  constructor(canvas) {
    this.scene    = createScene();              // Scene graph: holds all renderable objects
    this.camera   = createPerspectiveCamera();  // Perspective camera
    this.renderer = createRenderer(canvas);     // Renderer
    this.state    = { folded: true };           // Interaction state machine
  }
  addPart(geometry, material) {                 // Load a part/model
    const mesh = createMesh(geometry, material);
    this.scene.attach(mesh);
    return mesh;
  }
  onUserAction(event) {                         // Interaction drives state transitions
    this.state.folded = !this.state.folded;
    this.playTransition(this.state.folded);     // Trigger unfold/fold animation
  }
  frame(dt) {                                   // Main render loop
    this.updatePhysics(dt);
    this.updateAnimations(dt);
    this.renderer.render(this.scene, this.camera);
  }
}

After receiving model-generated code, it is worth focusing on three things: whether the state machine is clear (will consecutive operations cause state confusion), whether the main loop has a single responsibility (are physics, animation, and rendering decoupled), and whether resources are properly released when removed.

4. The Cost Perspective: Reading “Token Economics”

Another contribution of these cases is putting cost on the table. A quick summary follows (all data is self-reported by community users and for reference only):

Deliverable	Time	Token Consumption	Cost
Open-world city scene	~10 hours	~4 million	~$52
Foldable device interactive page	~1.5 hours	Not disclosed	~$12.5
3D launch short film	~5 minutes	Not disclosed	~$1.1
Cloth simulation / mechanical disassembly	~10 minutes each	Not disclosed	Not disclosed

A few notable patterns:

1. Cost correlates strongly with interaction complexity. The short film costs about 1/47 of the city scene. The difference is not in how polished the visuals are but in the number of states and iteration rounds.
2. Agentic coding consumes tokens in the millions. Each iteration re-carries the context, and input tokens accumulate repeatedly. When budgeting, do not estimate with a “single Q&A” mental model.
3. The actual cost of completing the same task is itself a selection metric. For the same “build a demo” task, total cost can differ several-fold across models, and this deserves a place in hands-on comparisons.

Engineering controls:

• Start small, then scale up: Trial-run a reduced version of the task (build one street before a whole city) to calibrate unit cost before releasing the budget.
• Freeze results in stages: Organize validated modules into files, and have later rounds reference summaries rather than the full context.
• Set explicit limits: Define a cost ceiling and a turn ceiling for the task to avoid the model getting stuck in unproductive repair loops.

5. Designing Your Own Validation Project: A Practical Checklist

If you want to test a model’s engineering ability yourself, consider designing tasks according to the following checklist:

1. Choose deliverables that can be seen and operated. Static pages do not reveal rendering or state management ability; pick goals with animation and interaction where possible.
2. Cover at least three capability dimensions: rendering (3D or motion), interaction (state machine/events), and data (loading/persistence).
3. Record metrics completely: wall-clock time, token consumption, cost, number of manual interventions, and rework rounds.
4. Give the model clear, verifiable acceptance criteria, for example:

Task goal: Generate an interactive web demo of a foldable device.
Hard constraints:
1) Single-file delivery, runs directly in a modern browser with no build step;
2) Supports both mouse click and touch gestures to trigger fold/unfold;
3) In both folded and unfolded states, the on-screen UI content must switch correctly.
Acceptance criteria:
1) Animation frame rate on a mid-range device is no lower than 60fps;
2) No state confusion after 50 consecutive operations;
3) Code is partitioned by module and key logic is commented.

1. Look at the code before drawing conclusions. Good visuals are only the passing line; whether the code structure is clear, naming is consistent, and exception paths are handled determines whether the output can enter your repository.

6. From Demo to Engineering: Practical Advice

Treat demos with respect and production with caution. The works shared in the community are mostly success stories under ideal conditions, and survivorship bias is unavoidable. If you want to put this kind of output into a real project, consider the following workflow:

1. Treat it as a “first draft”: Model-generated scene code is suitable as a prototype and scaffold. Architecture decisions and dependency choices should still be vetted by humans.
2. Enforce human review: Focus on performance hazards (memory leaks, runaway animation loops), browser compatibility issues, and the security of external dependencies.
3. Use standard pipelines for hosting and distribution: Static pages can be hosted on object storage or static website hosting services and distributed via CDN. Version management and canary releases should follow the team’s existing conventions.
4. Build a prompt template library: Templatize validated task descriptions, constraints, and acceptance criteria so team members can reuse them—saving tokens while producing consistent quality.

7. Conclusion

“The model got stronger” is a correct but useless judgment. The truly valuable question is: can it deliver something I can see, operate, and keep maintaining, within acceptable time and cost? Game prototypes, interactive web pages, mechanical disassembly—the methodology this round of community testing offers is actually quite plain: swap the evaluation standard from “score” to “deliverable,” and break the vague notion of “strong capability” into measurable numbers like time, tokens, cost, and rework count.

Whichever model you ultimately choose, this approach applies. Start with a small but complete deliverable, run the full loop of requirement–generation–acceptance–cost accounting, and then decide where it should sit in your and your team’s development workflow.