Evaluating Large Language Models on Complex Interactive Code Generation in Practice
This article explains how to evaluate large language models on complex interactive code generation, beyond single-function completion. It covers the shift from code completion to interactive prototype delivery, two representative front-end tasks (a draggable desktop window demo and a simplified real-time strategy game), the evaluation metrics and controlled variables to record, and engineering recommendations for stable results.
1. From Code Completion to Interactive Prototype Delivery: Raising the Evaluation Goal
As large language models move from code completion toward delivering interactive prototypes, the quality of a single generated function no longer shows whether a model has real engineering ability. More important questions are whether a model can understand abstract requirements, break down interaction logic, produce front-end pages that run, and keep a stable structure across multiple rounds of prompt iteration. This article starts from online experimentation and visual preview capabilities, and uses two typical tasks to discuss how to systematically evaluate model performance on complex interactive code generation.
Traditional code evaluation focuses on syntax correctness, algorithm pass rates, or single-file function completion. In real development, users often give composite requirements. Examples include “build a desktop-style web page with draggable windows and a 3D window switching effect,” or “convert a turn-based strategy game into real-time strategy gameplay with automatic research and automatic enemy targeting.” These tasks test the following abilities:
- Requirement understanding and abstraction: The model must identify interaction objects, state transitions, and implicit constraints from natural language.
- Front-end architecture: This includes layout, event systems, layer management, animation, and the rendering loop.
- Complex logic implementation: Examples include real-time game loops, path planning, and resource and research state machines.
- Multi-round iteration: When the first version falls short, can the model make targeted local fixes based on feedback instead of starting over?
- Engineering maintainability: Is the output modular, with clear naming, and easy to extend and test?
The evaluation goal should therefore move from “can the model generate code” to “can the model deliver a prototype that is interactive, verifiable, and iterable.” Combining an online experiment environment, code execution, and visual preview in one workspace is a key strength for this kind of evaluation. It shortens the generate, run, observe, and feedback loop, so developers can quickly find interaction defects and performance bottlenecks.
2. Breaking Down the Strengths: Live Preview, Iteration Loop, and Complex Logic Understanding
Three areas deserve attention.
First, online experimentation and visual preview. Showing code generation, run results, and page preview in one interface can greatly reduce verification cost. Static code review rarely catches front-end interaction problems such as drag boundary errors, layer occlusion, or animation stutter. Live preview exposes these problems quickly.
Second, the prompt iteration loop. Complex tasks seldom succeed on the first try. The model must respond to feedback such as “the window buttons do not look like semi-transparent glass,” “dragging goes out of bounds,” or “the research tree does not complete automatically” with targeted edits. During evaluation, check whether the model understands the scope of each change and whether existing features stay intact without regressing.
Third, turning abstract knowledge into runnable logic. A real-time strategy game requires the model to understand features such as a real-time loop, fair time for both sides, and no manual turn-ending button. An automatic research tree requires the model to understand the condition that research completes automatically when resources allow. Unit auto-targeting involves path planning and target selection. Simple templates cannot cover these requirements.
The same requirement may produce noticeably different results across model versions, sampling parameters, and concurrent loads. High-traffic periods can increase response latency, which lengthens iteration cycles and affects the user experience. For this reason, evaluation should not rely on a single trial. It should control variables, sample multiple times, and record version information.
3. Evaluation Design: Task Set, Metrics, and Controlled Variables
Use a standardized task set and record the following metrics:
- Functional correctness: Whether all requirements are implemented and whether any blocking defects exist.
- Interaction completeness: Whether dragging, clicking, switching, pausing, and selection operations form complete loops.
- Visual fidelity: Whether layout, layering, transparency texture, and animation rhythm match the description.
- Performance: First-screen time, interaction response, animation frame rate, and stability over long runs.
- Code maintainability: Module structure, state management, event decoupling, and extensibility.
- Iteration cost: The number of rounds needed to reach an acceptable result, prompt length, and the amount of manual correction.
For controlled variables, fix the model version, temperature, maximum output length, concurrency conditions, and browser environment. Run each task at least three times, and report the median of functional pass rates and subjective scores. Under high load, record queue wait time and retry failures so that service latency is not mistaken for a model capability problem.
For the evaluation environment, teams can use general cloud product categories, such as the container platform, log service, and monitoring and alerting provided by Baidu AI Cloud, to build a reproducible runtime and keep stress-test records. These are infrastructure examples only and do not imply any specific business partnership.
4. Case 1: Draggable Desktop Windows with 3D Switching
The task asks for a desktop-style web page in which windows can be dragged, with a classic desktop-style 3D window switching effect and buttons that have a semi-transparent glass texture. The task looks simple, but it involves several front-end details: the drag area on the title bar, pointer coordinate conversion, window layer management, boundary constraints, switch animation, and visual styling.
A reliable implementation usually has the following structure:
let topZ = 1;function makeDraggable(win) {let dragging = false;let offsetX = 0;let offsetY = 0;win.addEventListener('pointerdown', (e) => {if (!e.target.closest('.titlebar')) return;dragging = true;const rect = win.getBoundingClientRect();offsetX = e.clientX - rect.left;offsetY = e.clientY - rect.top;win.style.zIndex = ++topZ;win.setPointerCapture(e.pointerId);});win.addEventListener('pointermove', (e) => {if (!dragging) return;const x = Math.max(0,Math.min(e.clientX - offsetX, innerWidth - win.offsetWidth));const y = Math.max(0,Math.min(e.clientY - offsetY, innerHeight - win.offsetHeight));win.style.transform = `translate(${x}px, ${y}px)`;});win.addEventListener('pointerup', () => {dragging = false;});}
The key points of this code are that dragging starts only from the title bar, pointer events unify mouse and touch input, the window is raised while dragging, and coordinates are constrained to the viewport boundaries. The 3D switching effect can be built by combining CSS perspective, rotation, and scaling. During the animation, avoid triggering layout calculations frequently.
Common problems in evaluation include: a window dragged outside the viewport cannot be recovered, layer errors cause click-through, focus is lost during switch animations, and semi-transparent styles combined with shadows make the visuals muddy. In some cases, the model produces a page that looks right but has incomplete drag boundaries and layer management. In others, the visual details match the description more closely, but the code structure is scattered. In terms of experience, response latency is noticeably higher during peak hours, and results can still vary after several iterations. Evaluation should therefore record the prompt and output differences for each round, and should not draw conclusions from a single result.
5. Case 2: A Simplified Real-Time Strategy Game
The second task rewrites a classic turn-based strategy game as a simplified real-time strategy prototype for the web. Requirements include a real-time game loop, an AI-assisted research tree, automatic targeting so that all units find opponents, and additional control features. This task tests state management, real-time loops, pathfinding, and conditional logic.
The core of a real-time game loop is separating fixed-step updates from rendering:
const STEP = 1000 / 30;let last = performance.now();let acc = 0;function loop(now) {acc += now - last;last = now;while (acc >= STEP) {update(STEP / 1000);acc -= STEP;}render();requestAnimationFrame(loop);}
An AI-assisted research tree can be designed as a state machine. When resources meet the prerequisites, the system automatically selects the highest-priority available project. Automatic enemy targeting requires target selection and path planning, and A* or grid-based pathfinding can serve as a general solution. During evaluation, focus on two things. First, whether the model understands that “real-time” means fair time for both sides with no need to end turns manually. Second, whether “automatic research” actually connects to resource checks, rather than producing a static checkbox.
In testing, the ideal output correctly implements the real-time loop, automatic research, automatic enemy targeting, and control features. This suggests the model has a good abstract understanding of real-time strategy gameplay and conditional logic. Boundary tests are still needed. Check whether research pauses when resources are insufficient, whether units re-target after a target disappears, whether units have a retry mechanism when stuck on a wall, and whether state drift appears over long runs. These details determine whether the prototype can keep evolving.
6. Practical Experience and Engineering Recommendations
Overall, the experience can be summarized as: the strengths lie in the closed loop, and the difficulties lie in stability. Online preview and the iteration loop substantially improve prototype verification efficiency, and complex interactive tasks show the model’s logical understanding. However, high-concurrency response latency, output variation across versions, visual details, and performance limits remain the main pain points.
Engineering recommendations:
- Generate in stages: First ask for the data model and state machine, then the UI and interactions, and finally performance optimization. This reduces the risk of uncontrolled one-shot output.
- Build a requirements checklist: Write each requirement as a verifiable item and
