How to Evaluate Large Language Models for Interactive Application Generation

    Large language models can now generate code, page designs, and basic algorithms for games, data visualizations, and interactive experiments. However, visual polish and executable code do not guarantee correct behavior. This article presents a practical framework for evaluating whether a model can understand requirements, build consistent data structures, handle edge cases, implement valid interaction logic, and remain stable through iterative changes. It covers visual validation, spatial tasks such as Klotski and bowling, maze-based reinforcement learning, common failure modes, and a structured evaluation workflow.

    As large language models become better at code generation, interface design, and basic algorithm implementation, many developers are asking them to create mini-games, data visualization pages, and interactive experiments. In practice, however, the experience often falls short of the demonstrations. A page may look polished and the code may run, but rule validation, continuous interaction, or dynamic training can quickly expose logical errors.

    Evaluating whether a model is genuinely useful therefore requires more than checking whether it can produce code or create an attractive interface. More important questions include whether it can accurately understand requirements, establish correct data structures, handle edge cases, and keep the system stable after multiple rounds of modification. The following sections break down these capabilities through several representative tasks.

    1. Separate Generation, Execution, and Correctness

    In practical testing, results should be divided into at least three levels.

    1.1 Code Generation

    This is the most basic capability. A model can use a natural-language description to output page structure, styling code, and interaction logic. For example, it may generate a mini-game interface with animations or construct a simple reinforcement learning environment.

    However, producing code only shows that the model has organized text into a program-like structure. It does not prove that the code satisfies the requirements. Many models generate complete-looking code with extensive comments while still making mistakes in coordinate calculations, event binding, or state updates.

    1.2 Executability

    If the code loads, the page opens, and buttons respond to clicks, the result has reached a basic executable state.

    That still does not mean the functionality is correct. A page may render successfully while a canvas element covers important text. A button may be present but fail to update the application state. A training process may refresh numerical values continuously without actually changing the policy. These problems usually do not appear automatically during code generation and must be verified through hands-on testing.

    1.3 Task Correctness

    This is the most important standard. A task is complete only when it satisfies the required rules, interaction flow, and result constraints.

    For example:

    • Every piece in a Klotski puzzle must move legally and eventually reach the target position.
    • The bowling pin layout should follow basic spatial relationships, and the ball trajectory should be able to pass through the pin formation in a reasonable way.
    • In maze training, the agent must update its policy based on environmental feedback and develop stable path selection after multiple training rounds.
    • Visual design should support information display and interaction. It should not reduce usability through excessive decoration.

    2. Good Visual Design Cannot Replace Functional Validation

    Modern large language models are generally effective at generating gradient backgrounds, particle effects, lighting, and card-based layouts. For static pages or simple demonstrations, these capabilities can satisfy many entry-level requirements.

    However, visual polish can easily create the illusion that the functionality is complete. A page may have an attractive background, dynamic lighting, and refined buttons, while the main area is too dark to show the board, pins, or maze clearly. Alternatively, decorative layers may cover the interaction layer and prevent mouse events from being triggered accurately.

    When evaluating generated visual interfaces, check at least the following points:

    1. Information clarity
      Key elements need sufficient contrast. A dark visual style should not make text, boundaries, or status indicators difficult to read.

    2. Interaction safety
      Particles, shadows, and animations should be placed on separate layers and should not intercept clicks. When necessary, decorative elements should be excluded from mouse interaction.

    3. Responsive layout
      If the page works only inside the model’s preview window but overflows, overlaps, or loses its proportions in a standard browser or on a mobile device, it should not yet be considered reliable.

    4. Purposeful animation
      Animation should communicate state changes, user feedback, or process progress rather than simply adding visual effects. Excessive animation can also reduce performance and affect test results.

    3. Spatial Tasks Expose Common-Sense and Geometry Problems

    Klotski, bowling, and maze applications all involve spatial relationships, but they place different demands on the model.

    3.1 Klotski: Correct Arrangement Does Not Mean Correct Rules

    When generating a Klotski puzzle, a model can usually draw the board and pieces quickly, and it may arrange the target layout neatly. The difficult part is implementing legal movement.

    A reliable implementation must define:

    • The coordinate system used by the board;
    • The width, height, and occupied cells of each piece;
    • How to determine whether the destination area is empty before moving;
    • Whether an action would move a piece outside the board;
    • How mouse or touch dragging is converted into legal movement;
    • How completion is detected.

    If the model merely places the pieces in their target positions without implementing complete state transitions, the result is only a static illustration. Furthermore, if the model claims to have implemented a particular visual style but the final page does not reflect it, there is a disconnect between its explanation and its actual code.

    This can be understood as an “explanation hallucination”: the model produces a design description that sounds reasonable, while the code fails to implement all of it. Testing should therefore focus on runtime behavior and source-code logic rather than the model’s self-description.

    3.2 Bowling: Simple Graphics Still Require Geometric Constraints

    Bowling may appear to involve nothing more than drawing a ball and several pins, but it includes layout, spacing, collision, and perspective considerations.

    If the gaps between pins are too large, the ball will have difficulty hitting multiple targets. This is not merely an artistic issue; it means the layout parameters do not reflect common spatial rules. A model may produce an image that looks like bowling without understanding the relative distances between pins or the path of the ball.

    A parameterized design is more reliable than hard-coding every coordinate:

    1. Number of rows = n
    2. Horizontal spacing = d
    3. Vertical spacing = h
    4. Number of pins in row r = r
    5. Each pin position is calculated from the row, column, and spacing

    This approach provides several advantages:

    • The overall proportions are easier to adjust.
    • Individual pins are less likely to be misplaced.
    • Collision detection and trajectory calculation can be added more easily.

    The interface should also provide clear boundaries, aiming direction, and status feedback. Otherwise, objects may exist in the scene without being visible or understandable to the user.

    4. Maze Reinforcement Learning Provides a More Comprehensive Test

    A maze-based reinforcement learning task can test a model’s capabilities more thoroughly than a standard page or static mini-game. It combines environment modeling, agent actions, reward design, policy updates, the training loop, and synchronized visualization.

    4.1 Environment Layer

    The environment layer describes the maze map. A two-dimensional array can be used:

    1. 0: Walkable area
    2. 1: Wall
    3. 2: Start
    4. 3: Goal

    The map data must be the single source of truth. Maze rendering, collision detection, reward calculation, and agent-position display should all use the same data. If the visual layer uses one coordinate system while the action layer uses another, the agent may appear to be walking through a corridor while the logic treats the same position as a wall.

    4.2 Action Layer

    The action layer generally includes four actions: up, down, left, and right. Each action should perform the following steps:

    1. Read the current state.
    2. Select an action.
    3. Calculate the next position.
    4. Check whether the move is out of bounds or hits a wall.
    5. Return a reward based on the result.
    6. Update the agent state.
    7. Check whether the goal has been reached.

    If any of these steps is inconsistent, training may fail. Coordinate ordering deserves particular attention. Some implementations use “row, column,” while others use “x, y.” Without a consistent convention, visual and logical positions can easily become misaligned.

    4.3 Q-Value Update

    The core Q-learning update can be represented as:

    1. Q(current state, current action)
    2. =
    3. Q(current state, current action)
    4. +
    5. learning rate ×
    6. (
    7. immediate reward
    8. +
    9. discount factor × maximum Q-value of the next state
    10. -
    11. current Q-value
    12. )

    For a terminal state, the value of the next state should generally not continue to contribute to the calculation. If the model overlooks this, training values may grow abnormally.

    The balance between exploration and exploitation must also be handled. During the early stages of training, the agent should be allowed to try different paths. Later, it should increasingly select actions with higher estimated value. If it always acts randomly, training will not converge. If it always selects the current maximum from the beginning, it may become trapped in a poor path because early information is incomplete.

    4.4 Visualization Layer

    Visualization does more than present results; it also helps reveal errors. Useful information includes:

    • Current episode number;
    • Current step count;
    • Agent position;
    • Start and goal positions;
    • Q-values or action preferences for each cell;
    • Success rate or average number of steps.

    Displaying Q-values directly on floor cells is an intuitive way to observe whether value gradually propagates toward the goal. The displayed values must remain synchronized with the actual state, however. Updating text without updating the policy is not enough, nor is displaying only the final converged result while hiding the training process.

    5. Common Failure Modes and Their Root Causes

    Across repeated generations and revisions, problems usually fall into several categories.

    5.1 Requirements Are Too Broad

    “Build a good-looking maze game” is very different from “Use a fixed two-dimensional map, train an agent with Q-learning, and display changes in each cell’s value.” The first request mainly involves visual generation. The second requires the model to implement both algorithms and engineering details.

    Requirements should therefore specify:

    • What the inputs and outputs are;
    • Which rules must be satisfied;
    • How failures should be handled;
    • Whether the application must support restarting;
    • Whether training can be paused;
    • Which information the visualization should display.

    5.2 The Generated Code Is Too Large

    Asking for an entire project in one generation can cause modules to interfere with one another. A safer approach is to build the system in stages:

    1. Generate the static map.
    2. Add agent movement.
    3. Add the reward mechanism.
    4. Add Q-value updates.
    5. Add animation and data visualization.

    Validate the core functionality after each stage. This makes it easier to locate problems than searching through one large block of code.

    5.3 There Is No Automated Validation

    Visual inspection alone is not enough to find logical errors. Add simple assertions and tests, such as:

    1. Assert that the start position is walkable
    2. Assert that the goal position is walkable
    3. Assert that the agent never enters a wall
    4. Assert that every state has valid actions
    5. Assert that the average number of steps does not continuously worsen after training

    Even without a complete testing framework, these checks can substantially improve reliability.

    5.4 Multiple Coordinate Systems Are Not Unified

    This is a common problem in interactive applications. Map coordinates, canvas pixel coordinates, DOM positions, and mouse coordinates may all use different units. Without centralized conversion functions, elements can become misaligned.

    A better approach is to maintain one core coordinate model and calculate all presentation-layer positions from it. For example:

    1. Pixel X = column number × cell size
    2. Pixel Y = row number × cell size

    Coordinate conversions should not be manually duplicated across multiple modules.

    6. Building a More Scientific Evaluation Process

    To compare different models or prompts objectively, use a fixed test set rather than drawing conclusions from a single experience.

    Four categories of metrics are useful:

    6.1 Completion Rate

    Can the task run to completion? Does it support restarting? Does the page freeze or lose interaction during execution?

    6.2 Correctness

    Are the rules implemented correctly? Are spatial relationships reasonable? Does the training algorithm follow its formal definition?

    6.3 Stability

    Do repeated runs produce consistent results? Does modifying one local feature break other modules?

    6.4 Maintainability

    Is the code organized into modules? Are variable names clear? Are important parameters centrally managed? Can a developer modify the result easily?

    In practice, a slightly less impressive interface with stable logic and a clear structure is usually more valuable than a visually spectacular result that cannot be operated reliably.

    Conclusion

    Large language models are becoming increasingly capable of generating interactive applications, but the key risk remains the gap between presentation quality and engineering completeness. Lighting, animation, and page layout show that a model has some frontend-generation ability. Its real technical value is demonstrated by whether it can understand rules accurately, maintain data consistency, implement reliable interactions, and validate the result through testing.

    For developers, the most effective approach is not to give the model every requirement at once. Instead, break the task into small, verifiable modules and ask the model to provide implementation, tests, and error-handling strategies together. Only by combining visual design, business rules, algorithmic logic, and automated validation can developers determine whether a large language model is suitable for real-world application development.