Evaluating Complex Interactive Application Generation with Large Language Models

    This article examines practical approaches to evaluating large language models on complex interactive application generation tasks, such as draggable desktop windows and real-time strategy games. It outlines key evaluation dimensions, common failure modes, prompt iteration strategies, and engineering recommendations for reproducible testing. The focus is on moving beyond static page generation to assess a model's ability to handle state consistency, event-driven logic, and multi-module requirements.

    As general-purpose large language models continue to improve in code generation, front-end page assembly, and mini-game creation, an increasing number of model services are integrating prompt debugging, code preview, result comparison, and one-click execution into their online experimentation workspaces. The value of these workspaces extends beyond mere convenience—they place model capabilities within an observable, reproducible, and comparable interactive environment. For developers, the meaningful benchmark is no longer whether a model can generate a static page, but whether it can understand complex interaction logic, maintain state consistency, and deliver a complete, runnable implementation across multiple modules.

    In recent public evaluations, it has become common for anonymous model versions to be validated first, with official versions released later. Anonymous and official versions may differ in system prompts, sampling strategies, context truncation, safety policies, and concurrency scheduling. Therefore, performance observed in anonymous versions cannot be simply equated with that of official releases. In practice, increased traffic can extend inference queue times, raising first-token latency and overall generation duration. Moreover, iterative prompt refinements do not guarantee consistent improvements on the same task type, indicating that generation quality for complex interactive applications depends not only on model scale, but also on requirement specification, acceptance criteria, and evaluation methodology.

    Why Complex Interactive Applications Are Better Evaluation Samples

    Static page generation primarily tests layout, styling, and component composition. Complex interactive applications, in contrast, simultaneously test requirement decomposition, state machine design, event handling, real-time loops, pathfinding, input mapping, and rendering performance. Consider a “draggable desktop window” versus a “real-time strategy mini-game.” The former requires window layering, drag boundaries, minimize/maximize behavior, frosted-glass translucency, and 3D transition animations. The latter requires real-time time progression, resource and research state management, automatic unit targeting, hexagonal grids, speed controls, and keyboard-driven viewport navigation. Both task types carry high information density and expose a model’s weaknesses in long-chain reasoning and logic.

    When evaluating, avoid relying solely on final screenshots. Instead, record whether the generated artifact is runnable, whether the interactions are complete, how exceptional paths are handled, and whether the code structure is maintainable. A practical evaluation rubric may include: requirement coverage, interaction correctness, state consistency, visual fidelity, performance, code readability, and cost of secondary modification. Each dimension should be rated on a 1–5 scale by human reviewers, with prompts, generation timestamps, runtime logs, and screen recordings preserved for comparative analysis.

    Task One: Key Considerations for Desktop Window Interaction Generation

    This task type typically requires implementing window dragging, stacking management, button states, and 3D window transitions. Models often produce pages that merely “look right” but fail on specific details: windows dragging out of bounds, incorrect z-order, inability to restore from minimized state, animations that desynchronize from window state, and frosted-glass effects that turn muddy when multiple layers overlap. Evaluation should focus on whether the window state model is clearly defined.

    A recommended state design is as follows:

    1. const windowState = {
    2. id: "win-1",
    3. x: 120, y: 80, width: 640, height: 420,
    4. zIndex: 10,
    5. minimized: false,
    6. maximized: false,
    7. active: true
    8. };
    9. function focusWindow(state, allWindows) {
    10. const maxZ = Math.max(...allWindows.map(w => w.zIndex), 0);
    11. state.zIndex = maxZ + 1;
    12. allWindows.forEach(w => w.active = false);
    13. state.active = true;
    14. }

    Drag logic should use pointer events to unify mouse and touch input, with boundary constraints applied during movement. The 3D transition view should maintain its own state based on window snapshots or thumbnail views, avoiding direct reuse of the main view so that animation completion does not cause state corruption. Frosted-glass translucency can be achieved through background blur, border highlights, and shadows, but performance must be considered—avoid placing expensive filters on large surface areas.

    Task Two: Comprehensive Difficulty of Real-Time Strategy Game Generation

    Real-time strategy games serve as excellent comprehensive benchmarks. The requirement to “convert turn-based to real-time” is not simply about removing a turn button; it demands that the model understand time fairness in a real-time system. Both sides progress on the same timeline, with unit movement, resource growth, research completion, and combat resolution all driven by a main loop. The model must establish a stable game loop and decouple rendering from logic updates.

    A typical main loop can be written as:

    1. let last = performance.now();
    2. let speed = 1;
    3. function loop(now) {
    4. const dt = Math.min((now - last) / 1000, 0.05) * speed;
    5. last = now;
    6. updateResearch(dt);
    7. updateUnits(dt);
    8. updateCombat(dt);
    9. render();
    10. requestAnimationFrame(loop);
    11. }

    The “AI-assisted research tree” tests intent understanding. Such a feature does not exist in the base game by default; the model must add a checkable assistant system that autonomously selects research items when resources permit. The research tree should be modeled as a dependency graph, with the assistant choosing unlockable nodes based on priority and resource constraints, rather than arbitrarily deducting resources. Automatic unit targeting tests pathfinding and target selection; grid-based pathfinding with target scoring is recommended to avoid full-map searches every frame.

    Hexagonal grids require coordinate conversion. A common approach uses axial or cubic coordinates; the following is a schematic conversion from axial coordinates to pixels:

    1. function hexToPixel(q, r, size) {
    2. const x = size * Math.sqrt(3) * (q + r / 2);
    3. const y = size * 1.5 * r;
    4. return { x, y };
    5. }

    The speed slider should adjust the main loop time scale from 1x to 10x. Keyboard controls for viewport panning should shift the camera offset rather than moving all units. City and unit icons can use emoji or system icons, but must remain readable and scale consistently.

    Results Observation and Prompt Iteration Methodology

    From practical observations, anonymous and official versions may show notable differences in visual and interactive completeness on the same task, and two rounds of prompt iteration do not necessarily outperform earlier versions. Possible causes include truncation under concurrent load, context compression, sampling randomness, and changes in system policies. Therefore, evaluations should not conclude from a single run. It is recommended to sample each task at least 3–5 times, recording the median outcome and failure reasons.

    Prompt iteration should follow a phased strategy: first ask the model to output requirement decomposition and a module list, then generate the base framework, and finally fill in interactions one by one. Write acceptance criteria directly into the prompt, such as “drag must not go out of bounds,” “minimized windows must be restorable,” “the research assistant must not deduct resources when insufficient,” and “the speed slider must affect both unit movement and research progress.” Keep the input version fixed, record generic sampling settings such as temperature, and preserve generated artifacts alongside runtime logs to avoid subjective comparisons based solely on screenshots.

    Engineering Recommendations for Deployment

    In a cloud environment, generated artifacts, prompts, runtime logs, and evaluation scores can be stored separately in object storage and log services. Container platforms can quickly reproduce the runtime environment, while monitoring and alerting track generation latency and error rates. For high-concurrency trial scenarios, use streaming output, task queues, and cache degradation to avoid long user wait times. Ultimately, the evaluation of complex interactive application generation should return to three standards: runnability, verifiability, and maintainability. A working artifact is only the starting point; interaction completeness and the cost of secondary modification are what truly determine delivery quality.