Evaluating Interactive Visual Generation Capabilities of Large Models: Methods and Practices

    As general-purpose large models continue to improve, more developers are evaluating how well they perform in front-end visual generation, interactive prototype building, and lightweight game generation. This article argues that subjective impressions alone are insufficient and proposes a structured evaluation approach that moves beyond visual appearance to assess layout generation, state management, physics simulation, visual consistency, accessibility, and engineering maintainability. It outlines evaluation principles, typical test scenarios, automated checks combined with human review, common failure patterns, and practical engineering suggestions for determining whether a new model capability is truly reliable.

    Introduction: From “Can Generate” to “Can Be Evaluated”

    With continued improvements in general large-model capabilities, more developers are paying attention to how well large models perform in scenarios such as front-end visual generation, interactive prototype construction, and lightweight game generation. A typical feature of these tasks is that a user provides a natural-language description, and the model must output a runnable page or interactive program. The result should not only look reasonable visually, but also work logically, respond to interaction, and remain maintainable from an engineering perspective.

    Therefore, when asking “What new highlights should we look for after a new model is released, and what is the usage experience like?”, a more rigorous analysis should not rely only on subjective impressions. Instead, it is better to build a structured evaluation method. The object of evaluation can be generalized from “page generation” to “interactive visual generation,” covering layout generation, state management, physics simulation, visual consistency, accessibility, and engineering maintainability. This approach helps avoid being misled by a single demo and makes it easier to identify the model’s capability boundaries and directions for improvement.


    1. Core Challenges of Interactive Visual Generation Tasks

    Generating a visual interface with a large model may look like simply “writing code,” but it actually involves challenges at multiple levels.

    1.1 Semantic Understanding and Structural Reconstruction

    User descriptions are often short, such as “generate a playable sliding puzzle” or “generate an interactive page with particle effects.” The model needs to translate them into page structure, component hierarchy, style rules, and interaction logic. A key difficulty is that natural-language descriptions often contain implicit constraints, such as:

    • Whether the page needs to adapt to different screen sizes;
    • Whether the main visual area must be centered;
    • Whether controls must be usable in the initial state;
    • Whether interaction feedback must be timely and obvious;
    • Whether complex effects can be gracefully degraded into more reliable implementations.

    If a model simply stitches together visual effects while ignoring structural completeness, the result may be “visually present, but functionally absent.”

    1.2 Interaction Logic and State Management

    Many visual generation tasks are not static pages but applications with changing states. For example, a collision-based game needs to manage the ball’s position, velocity, collision results, reset logic, and end conditions. The model must not only generate code but also maintain a runnable state machine.

    Common problems include:

    • Incomplete event binding;
    • Animation loops that are not properly stopped;
    • State that is not cleared after reset;
    • Win or failure conditions that cannot be triggered;
    • Element position updates that are not synchronized with rendering.

    These issues are difficult to detect through a single screenshot or static inspection and must be exposed through actual interaction testing.

    1.3 Physics Common Sense and Spatial Logic

    For tasks involving collision, scrolling, gravity, trajectories, or grid alignment, the model needs some basic physics common sense. This does not mean the model must implement an accurate physics engine; rather, the result should match basic intuition. For example:

    • A ball should bounce in a reasonable direction after collision;
    • Objects should not have obviously unreasonable initial spacing;
    • Grid-based puzzles should move according to grid cells instead of becoming randomly misaligned;
    • Dragged elements should be constrained within valid areas;
    • Collision detection should not show obvious pass-through behavior.

    If the generated result only “looks about right” visually but becomes impossible to complete during interaction, its experience score should be significantly reduced.

    1.4 Visual Consistency and Style Constraints

    When users request a certain style, the model needs to keep the visual expression consistent with that requirement. For example, if a user wants a clean, technology-oriented interface, the result should not contain too many decorative elements; or if a user wants a high-contrast dark interface, text and backgrounds should still be easy to distinguish.

    These issues can usually be attributed to insufficient prompt-following ability. A model may claim in its explanation that it used a certain style, but the actual output does not reflect it. This indicates a gap between the generated result and the internal explanation. Therefore, “style consistency” should be evaluated as a separate dimension.

    1.5 Runnability and Engineering Maintainability

    What developers ultimately need is runnable, modifiable, and extensible code, not effects that can only be shown once. In evaluation, beyond checking whether the page opens, we should also check:

    • Whether the console reports errors;
    • Whether there is a large amount of redundant code;
    • Whether the result overly depends on external resources;
    • Whether necessary comments are missing;
    • Whether it is easy to replace data or modify styles later;
    • Whether it runs stably in common browser environments.

    This dimension determines whether the generated result can enter a real development workflow instead of remaining only at the demo stage.


    2. Basic Principles for Building an Evaluation Framework

    To make evaluation results more valuable, several principles can be followed.

    2.1 Realism of Tasks

    Evaluation tasks should be close to real development scenarios rather than overly idealized. Examples include:

    • Form page generation;
    • Data card display;
    • Sliding-puzzle interactions;
    • Collision-based ball movement;
    • Simple dashboards;
    • Animation switching effects;
    • Responsive layout pages.

    These tasks can cover structured layout, state changes, animation rendering, and user interaction.

    2.2 Multi-Dimensional Scoring

    Conclusions should not be based only on “looks good” or “runs.” It is recommended to score comprehensively across dimensions such as:

    Dimension Focus Check Method
    Visual completeness Layout, color, hierarchy, alignment Screenshot review, manual scoring
    Interaction usability Click, drag, reset, state changes Manual operation or automated scripts
    Logic correctness Conditional logic, flow control, state recovery Code review and runtime testing
    Physical reasonableness Collision, trajectory, spacing, gravity Repeated runs and observation
    Style consistency Whether it matches the prompt Manual comparison
    Engineering maintainability Clear structure, low coupling, low redundancy Code review
    Error handling Invalid input, reset, empty state Boundary testing

    2.3 Reproducibility

    For the same prompt, input, and environment, results should be run multiple times whenever possible, and the average or distribution should be recorded. Generative tasks have a degree of randomness, so a single result is not enough to represent overall capability. If the platform supports it, random seeds can be fixed; otherwise, multi-round sampling can be used.

    2.4 Progressive Complexity

    Evaluation tasks should be designed from simple to complex. For example:

    • Beginner tasks: Generate a page with buttons and input fields;
    • Intermediate tasks: Generate a card component with switching animations;
    • Advanced tasks: Generate a sliding puzzle that can be completed;
    • Challenge tasks: Generate an interactive scene with collision, scoring, and reset mechanisms.

    This layered approach helps locate how model capabilities change at different difficulty levels.


    3. Typical Evaluation Scenarios

    Below are two representative abstract evaluation scenarios used to observe how models perform in front-end visual generation and interaction logic.

    Scenario 1: Sliding Grid Puzzle

    This scenario requires the model to generate a puzzle interface that can move within a grid, maintain an orderly layout, and determine whether the puzzle is complete.

    Key Evaluation Points

    1. Whether the grid structure is complete
      All cells should have clear boundaries and identifiable positions, without overlap or misalignment.

    2. Whether the movement logic is correct
      When users click or drag, puzzle pieces should move only within valid areas and should not pass through other pieces.

    3. Whether the initial state is reasonable
      Puzzle pieces should start in a solvable state, not randomly stacked or impossible to complete.

    4. Whether the completion state can be judged
      After puzzle pieces are arranged in the target order, a hint or state change should be triggered.

    5. Whether interaction feedback is obvious
      After user operations, there should be visible position changes, selected states, or completion prompts.

    Example Scoring

    • Visual structure: 20 points;
    • Movement logic: 25 points;
    • Initial solvability: 15 points;
    • Completion detection: 20 points;
    • Interaction feedback: 10 points;
    • Code maintainability: 10 points.

    The advantage of this task is that it evaluates the model’s understanding of spatial relationships, state management, and visual consistency at the same time.


    Scenario 2: Collision-Based Ball Movement

    This scenario requires the model to generate an interactive page with balls, obstacles or target areas, scoring, and reset mechanisms.

    Key Evaluation Points

    1. Whether collision logic is reasonable
      Collisions between the ball and boundaries or obstacles should follow basic physical intuition, such as bouncing, stopping, or triggering scoring.

    2. Whether the initial layout is reasonable
      Objects should not have obviously unreasonable gaps, and they should not start in a colliding state.

    3. Whether the movement process is smooth
      Animation frame updates should be stable, without stuttering, skipped frames, or element flickering.

    4. Whether the reset mechanism is reliable
      After the user clicks reset, all states should return to their initial values, including position, velocity, score, and end state.

    5. Whether visual visibility meets standards
      Backgrounds, balls, trajectories, and buttons should have sufficient contrast, avoiding situations where “the page is too dark to see anything clearly.”

    Example Scoring

    • Visual visibility: 15 points;
    • Motion smoothness: 20 points;
    • Collision reasonableness: 25 points;
    • Scoring and reset logic: 25 points;
    • Page structure clarity: 15 points.

    This type of task can effectively expose shortcomings in a model’s physics common sense, state consistency, and level of detail completion.


    4. Combining Automated Checks with Human Evaluation

    Because interactive visual generation tasks are multifaceted, relying only on automation or only on human evaluation is unlikely to cover all problems. A reasonable approach is to combine both.

    4.1 Automated Checks

    Automated checks are suitable for quantifiable and repeatable issues, such as:

    • Whether the page loads normally;
    • Whether exceptions appear in the console;
    • Whether key elements exist in the page structure;
    • Whether buttons, inputs, and canvas areas are visible;
    • Whether the page has obvious blank areas;
    • Whether text color and background color meet basic contrast requirements;
    • Whether the reset button triggers state changes;
    • Whether there are long-running animation loops with no response.

    A simplified general check approach is shown below:

    1. function runBasicChecks(page) {
    2. const errors = [];
    3. if (!page.querySelector('#main-area')) {
    4. errors.push('Missing main interaction area');
    5. }
    6. const visibleElements = Array.from(
    7. page.querySelectorAll('button, input, canvas, [role="button"]')
    8. );
    9. if (visibleElements.length === 0) {
    10. errors.push('The page has no interactive elements');
    11. }
    12. if (!page.querySelector('[data-status]')) {
    13. errors.push('Missing status display node');
    14. }
    15. return errors;
    16. }

    This type of check cannot replace human experience, but it can quickly identify obvious problems in the first round of screening.

    4.2 Human Evaluation Focus

    Human evaluation is more suitable for judging the following issues:

    • Whether the visual effect is natural;
    • Whether interaction feedback meets expectations;
    • Whether there is physical behavior that contradicts intuition;
    • Whether the page style matches the prompt;
    • Whether users can understand the operation method on first use;
    • Whether there are subtle issues that affect the experience.

    For example, even if a generated result can run, problems such as buttons that are too small, click areas that are not obvious, or success prompts that are too hidden can significantly affect usability, even if automated checks do not catch them.


    5. Common Problems and Root-Cause Analysis

    In practical evaluation, common defects are often not isolated errors but the combined result of several capability gaps.

    5.1 Disconnect Between Visual Generation and Functional Logic

    Some results have a complete visual layer, but the interaction logic does not form a closed loop. The root cause is usually that the model interprets the task as “generate a good-looking page” rather than “generate an application that can complete a task.”

    5.2 Poor Experience Caused by Insufficient Physics Common Sense

    In collision-based scenarios, if object spacing is unreasonable or collision responses are unnatural, users will immediately notice the problem. This indicates that the model still has gaps in understanding spatial relationships and basic motion rules.

    5.3 Unstable Style Following

    A prompt may request a certain visual style, but the generated result only retains a few keywords, such as color changes or simple decorations. This reflects insufficient ability to decompose complex style instructions.

    5.4 Low Contrast and Missing Accessibility

    Some generated results pursue visual effects by using dark backgrounds, low-brightness elements, and weak contrast, making key information invisible. Readability should be treated as a basic requirement in evaluation, not a bonus item.

    5.5 Overly Bloated Code Structure

    Some generated results can run but contain many repeated styles, unused variables, or excessively nested structures. This increases future maintenance costs and makes secondary development harder.


    6. Engineering Optimization Suggestions

    To improve the usability of models in interactive visual generation tasks, we can start from prompt design, task decomposition, automated validation, and result governance.

    6.1 Use Structured Prompts

    Prompts should clearly state the goal, constraints, and acceptance criteria. For example:

    1. Please generate a runnable sliding puzzle page with the following requirements:
    2. 1. Use a clear grid layout;
    3. 2. All cells should start in a solvable state;
    4. 3. Users should be able to move puzzle pieces by clicking or dragging;
    5. 4. Show a success message when the puzzle is completed;
    6. 5. Provide a reset button;
    7. 6. Text and background should have sufficient contrast;
    8. 7. Do not depend on external network resources;
    9. 8. If complex animations are difficult to implement, prioritize correct interaction.

    This type of prompt is better than simply saying “make a sliding puzzle,” because it reduces deviation in the result.

    6.2 Break Large Tasks into Smaller Tasks

    Complex interactions can be split into multiple stages:

    1. First generate the static layout;
    2. Then add data structures and states;
    3. Next implement movement logic;
    4. Finally add completion detection and reset functionality.

    This step-by-step approach reduces omissions caused by asking the model to generate too much at once.

    6.3 Establish Automated Acceptance Rules

    For repeatable tasks, acceptance conditions can be predefined:

    • The page loads without errors;
    • Core elements exist;
    • The reset button can be clicked;
    • The status display node updates;
    • There is no obviously invisible text;
    • The page adapts to common viewport sizes.

    These rules can be accumulated into a test checklist, making it easier to continuously evaluate different models or versions.

    6.4 Pay Attention to Runtime Environment Isolation

    In cloud environments, capabilities such as large-model inference, result storage, logging services, and monitoring/alerting can be combined as common engineering components. This makes it possible to track changes in generated results and quickly locate whether a problem is caused by the prompt, model output, runtime environment, or front-end compatibility.

    6.5 Keep a Human Review Step

    For product prototypes aimed at end users, automated checks can only serve as initial screening. Whether the result is ultimately usable still requires developers to verify it through real operation. This is especially important for interaction details, visual comfort, and exception-state handling, where human judgment remains irreplaceable.


    7. Evaluation Conclusion: How to Judge Whether It Is “Worth Watching”

    For a newly released large-model capability, what matters most is not whether a single demo is impressive, but whether the model has stable generation boundaries and an explainable capability profile. For interactive visual generation tasks specifically, the following points are worth focusing on:

    1. Whether it can generate structurally complete pages
      This is a foundational capability that determines whether the result is usable.

    2. Whether it can complete basic interaction loops
      Whether users can operate the interface, see feedback, and complete the task is central to the experience.

    3. Whether it has basic physics and spatial common sense
      This determines whether tasks involving movement, collision, and layout are credible.

    4. Whether it follows visual style constraints
      This shows whether the model can stably understand complex prompts.

    5. Whether it has engineering maintainability
      This determines whether the generated result can enter a real development workflow rather than being only a one-time display.

    Overall, evaluating interactive visual generation capability should evolve from “visual impression” to a comprehensive assessment of structure, interaction, logic, common sense, and engineering quality. Only by establishing clear test sets, scoring dimensions, and review mechanisms can we more objectively judge model capability boundaries and provide a basis for future prompt design, product integration, and development workflow optimization.