Evaluating LLM Coding Ability with Complex Interactive Tasks
A model-agnostic method for judging LLM coding ability with high-density interactive tasks: a desktop-style multi-window page and a simplified real-time strategy prototype. It covers what to accept, how to score and iterate, how to write prompts, and how to separate generation delay from runtime correctness.
Complex front-end work and lightweight game prototypes are high-information tasks for testing an LLM’s overall coding ability. They require visual fidelity, state management, an event loop, rule abstraction, and adjustable interaction at the same time. They expose misunderstanding, loose structure, and regression across iterations more clearly than “build a static page.” The framework below does not depend on a specific model or platform. It can be used directly for internal selection, prompt engineering, and acceptance.
Why use integrated interactive tasks
Code completion, function implementation, and documentation Q&A mainly test local correctness. An interactive prototype requires the model to turn a natural-language request into a running system: layout and style, component state, time progression, entity behavior, input mapping, and boundary conditions must all hold together. If any part exists only in the wording, the page can look complete and still fail as soon as someone uses it.
A task works as a benchmark when it meets four conditions: each requirement can be checked one by one; correct behavior does not depend on hidden context; the result can be observed directly in a browser; and failure modes can be attributed, such as missing styles, logic that was never wired up, or performance that makes the result unusable. Difficulty should sit in the range of “one file or a few files, but cross-module reasoning is required.” That avoids template assembly without demanding a full engine on the first attempt.
Task 1: A desktop-style interactive page
The first probe targets interface understanding and front-end implementation quality. A neutral request is enough: recreate the window-management feel of a classic desktop operating system as a multi-window interface that can be dragged inside the page, and add a window preview similar to switching in a three-dimensional space. Acceptance should not stop at how a screenshot looks. Each action should be tried.
Visual fidelity means checking that the window title bar and controls such as minimize, maximize, and close are recognizable and stay in stable positions; that the background and windows have depth, blur, or translucent glass-like layering; and that shadows and borders remain distinguishable when windows overlap. Interaction correctness means checking that dragging follows the pointer, that position is kept after release, that the most recently used window comes to the front, and that a preview switch returns to the matching window. Structural quality means checking that styles are managed in one place, that events are not bound repeatedly, and that window data is described by unified state rather than coordinates hardcoded across unrelated fragments.
This kind of task often produces “stacked visual keywords”: the model writes blur, rounded corners, and gradients, but controls cannot be clicked, the drag coordinate system is wrong, or the switch animation has no correspondence with window identity. Evaluation notes should therefore separate “looks like it” from “can be used.” Prompts should state the coordinate system, stacking rules, and the behavior of each control, instead of offering only style adjectives.
If the first result is structurally messy, later turns should change the failing point incrementally rather than rewriting the whole style description. For example, constrain one thing at a time: dragging updates the window’s top-left coordinates from pointer movement; the switch view maintains a list of window identifiers; clicking a control must not start a drag. Changing only one failed dimension per round makes it possible to tell whether the model understood more or whether earlier context was overwritten. A common regression is a longer later response whose glass effect and buttons are worse. Prompt inflation did not strengthen the constraints and may have overwritten details that were already working.
Task 2: A real-time-strategy web mini-game
The second probe turns abstract rules into a playable prototype and is suited to testing integrated reasoning. The request can be: implement a simplified real-time strategy in the browser, not a turn-based game; both sides act on the same timeline and no manual end-of-turn is required; the map uses regular hexes; units automatically seek opponents or strongholds; a speed slider runs from 1x to 10x; arrow keys pan the view; cities, units, and other major objects have recognizable icons, and character icons are acceptable; add an optional automatic research assistant that, when resources allow, completes research by a preset priority instead of waiting for the player to click each item.
These clauses test different layers.
“Make it real-time strategy” requires the model to understand the genre: there is no turn button, time is fair to both sides, and actions keep resolving inside a loop. Renaming a button while still advancing by turns should be marked as a failure.
“Automatic research assistant” is not a default feature of a common template. The model must recognize a new subsystem outside the base rules: a resource threshold, a research queue, a toggle, and automatic spending must all exist. A panel that does not actually deduct resources or advance technology is an interface pass and a logic failure.
“Units automatically find enemies” involves targeting. Hex neighbors, distance, or simplified pathfinding needs at least one consistent rule. Units should move toward an opponent or a city, not wander at random or oscillate in place. Industrial-grade pathfinding is not the goal, but every step must be explainable and reproducible.
Controls check requirement coverage: speed must actually change the simulation step or interval, not only the number on the slider; panning the view must not confuse entity coordinates with camera offset; icons must be bound to object types and must not disappear after a refresh.
Evaluation process and records
Use the same requirements, the same runtime environment, and the same acceptance checklist. Otherwise prompt length or browser differences are mistaken for model differences. The process has five steps.
- Freeze the requirements. Write each one as an observable result, for example: “At 10x speed, unit displacement frequency is about ten times the 1x rate. Shortening the interval or enlarging the step is acceptable, but the chosen method must be stated.”
- Generate once. Record whether it runs on the first try, whether the console reports errors, and whether the first screen is interactive.
- Score against the checklist. Mark visual fidelity, rule correctness, input mapping, state persistence, and code maintainability as pass, partial pass, or fail, and save the failure itself, not only a subjective impression.
- Iterate within a limit. Use at most two or three rounds, and add constraints only for items that failed. If the score drops, keep the last usable version and decide whether the cause was conflicting constraints or forgotten context.
- Separate load and latency. When many people are using the system, or the generation context is very long, a slower response does not by itself mean worse logic, but it does affect iteration speed. Record generation time separately from frame rate and logical correctness. Do not reject a working feature only because generation was slow, and do not ignore an unusable wait just because the feature is correct.
When comparing models or prompt versions, compare only passed items and regressions on the same checklist. “Looks closer to the request” should be pinned to identifiable evidence such as button shape, transparency, layering, and drag feel. For the game, pin it to fair time, whether automatic research actually settles, and whether enemy seeking is stable. Do not replace checklist items with an overall feeling.
How to write the prompt
An effective prompt gives the system boundary first, acceptance clauses second, and implementation constraints last. The boundary includes running as a single page, not depending on external assets, and using native page technologies or the team’s standard front-end stack. Number the clauses and state prohibited behavior, for example: do not fake real time with an end-turn button, and do not show a research panel without automatically spending resources. Keep implementation constraints restrained: centralize state, separate the loop from rendering, and use either axial or cube coordinates for hexes consistently throughout. Specifying many libraries and flashy effects too early crowds out attention that should go to the rules.
A schematic requirement skeleton can look like this:
Goal: a playable simplified real-time strategy prototype in the browserMap: regular hexes; camera can be panned with arrow keysTime: both sides share one clock; speed slider 1–10Units: automatically approach the nearest enemy unit or strongholdResearch: optional assistant; when resources are sufficient, complete items from the queue automaticallyInterface: objects use character icons; key state is visibleAcceptance: demonstrate each clause; if it fails, state which clause was not met
On later turns, attach the concrete failure, such as “after the slider changes, the internal interval is still fixed.” That is more effective than repeating “make it more like a game.”
Interpreting results and putting them into engineering use
One evaluation does not establish a model’s full ability, but it can show whether the model fits a class of delivery: a high-fidelity interface prototype, a rule-dense small tool, or iterative development that must keep old behavior across several revisions. If interface clauses pass reliably while rule clauses keep failing, use that model for visual drafts and fill in the rules manually or with stricter tests. If the first round is logically correct and later rounds get worse, the problem may be the prompting strategy: shorten each increment, lock clauses that already passed, and require that behavior not mentioned stay unchanged.
Before generated code enters a repository, a person still needs to review event leaks, infinite loops, out-of-range coordinates, and resource counts. Automatic research and automatic enemy seeking are continuous settlement logic. Regression-test them from a small set of fixed starting positions: after a set number of steps at the same speed, positions, resources, and technology state should be predictable. On performance, separate waiting for generation from running the page. When heavy concurrent use slows generation, the evaluation should note the environment rather than writing queue delay down as an algorithm defect.
For a team, the value of this method is turning “does it work well” into items that can be reviewed again: whether the requirement was understood, whether subsystems actually run, whether iteration preserves behavior, and whether the experience is acceptable. Accumulating results against the same checklist makes it possible to compare prompt designs, model versions, and usage scenarios instead of stopping at a single screenshot and a subjective preference. If a cloud platform only provides general model invocation, logs, and monitoring, the evaluation itself should still center on the task checklist and runtime results. Platform capability should not be mistaken for model capability.
