Evaluating Large Language Models’ Complex Interactive Generation Capabilities
As large language models move beyond text generation toward executable code generation, evaluating answer accuracy alone is no longer sufficient. This article presents a neutral, reproducible framework for assessing complex interactive web and lightweight game prototypes, covering requirement understanding, state management, interaction design, path planning, exception handling, code organization, structured prompting, quantitative scoring, and code-level acceptance testing.
Evaluating Large Language Models’ Complex Interactive Generation Capabilities
As large language models move from text generation to executable code generation, answer accuracy alone is no longer enough to assess their engineering value. Tasks that more closely resemble real-world development often combine requirements analysis, state management, interaction design, path planning, exception handling, and code organization. Complex interactive web pages and lightweight game prototypes provide high-density test scenarios for evaluating these combined capabilities.
This article does not recommend any specific model or product. Instead, it discusses how to design a reproducible, measurable, and extensible evaluation framework for complex interactive generation from a general technical assessment perspective.
1. Why Complex Interactive Tasks Are More Discriminating
Generating a simple page typically requires only an understanding of layout and styling, such as creating a form, card, or navigation bar. Complex interactive tasks require the model to maintain consistency across multiple dimensions:
- Requirement abstraction: When a user asks to “convert a turn-based game into a real-time strategy game,” the model must understand the defining characteristics of real-time strategy, including continuous time progression, fair time allocation for both sides, automated unit behavior, and parallel resource and production systems.
- State modeling: Window dragging, layer switching, unit pathfinding, and research queues are not isolated functions. They share a common state space. The model must design clear data structures and update flows.
- Interaction completeness: The result must not merely display information; it must also be operable. Button feedback, draggable windows, synchronized selection states, and boundary constraints are all part of acceptance testing.
- Code maintainability: A runnable one-off demo is only the baseline. Modularity, event decoupling, and the absence of obvious memory-leak risks determine whether the generated result can support further development.
Evaluation should therefore examine the complete path from requirements to implementation, rather than focusing only on the final screenshot.
2. Evaluation Principles: Neutral, Layered, and Reproducible
To reduce the influence of subjective impressions, evaluation should follow four principles:
- Neutrality: Design tasks around general capabilities instead of subjective descriptions of a particular product.
- Layering: Divide tasks into basic, combined, and open-ended capabilities.
- Reproducibility: Fix the prompt version, runtime environment, initial code constraints, and acceptance scripts.
- Blind evaluation: Keep evaluators unaware of which model produced a result to reduce brand bias.
The basic capability layer can assess window dragging, button clicks, and list rendering. The combined capability layer can assess multi-window switching, real-time state updates, and automatic pathfinding. The open-ended layer can include realistic extensions, such as adding an AI assistant that automatically completes research when resources allow.
3. Typical Use Cases
3.1 Desktop Window Interaction Prototype
A task might ask the model to generate a draggable desktop window interface that supports switching between multiple windows and provides a spatially oriented window-switching view. Acceptance criteria include:
- Whether press, move, and release events are handled completely;
- Whether dragging is constrained to the visible area;
- Whether the z-index or layer order is correct when multiple windows are activated;
- Whether window state is preserved when switching views;
- Whether visual elements such as transparency, shadows, and rounded corners remain consistent.
Although this task appears to involve basic frontend development, it tests the model’s ability to handle pointer events, coordinate conversion, state synchronization, and visual consistency at the same time.
3.2 Converting a Turn-Based Game into Real-Time Strategy
Converting a turn-based strategy game into a real-time strategy game is a typical high-abstraction requirement. The model must understand that:
- Time progression changes from clicking a “next turn” button to a continuous tick loop;
- Production, movement, and combat can occur in parallel;
- Resource settlement and unit behavior must use a unified clock;
- Both sides must receive fair time allocation, without structural advantages caused by operation frequency.
If the model only hides the “next turn” button without introducing a tick loop and automated behavior scheduling, it has failed to understand the requirement.
3.3 AI-Assisted Research Tree
“Add an AI assistant that automatically completes research when resources are available” is an open-ended extension request. Acceptance criteria include:
- Whether an AI assistant toggle is available;
- Whether the assistant automatically selects research items under resource constraints;
- Whether duplicate queue entries are prevented;
- Whether the assistant works alongside or is mutually exclusive with manual research;
- Whether the UI reports the assistant’s current automated decision.
The key is not simply to “click a button automatically,” but to abstract resource constraints, priorities, and queue state.
3.4 Automatic Unit Targeting and Control
Having all units automatically search for opponents and cities involves path planning, target selection, conflict handling, and state-machine design. Control functions may include pause, resume, speed adjustment, selection, and deselection.
During evaluation, observe whether the model separates pathfinding logic from rendering logic and whether it handles disappearing targets, blocked paths, and overlapping units.
4. Prompt Iteration Methods
High-quality evaluation requires prompts that can be iterated, rather than a one-time natural-language description. A useful structure is:
Goal – Constraints – Acceptance Criteria – Feedback
Goal: Generate a runnable complex interactive web prototype.Constraints: Use only native frontend technologies; manage all state centrally; do not depend on external services.Acceptance criteria:1. Windows can be dragged without leaving the visible area;2. Window state is preserved when switching between multiple windows;3. A real-time loop drives unit behavior;4. Automatic research is constrained by available resources;5. Provide pause and resume controls.Feedback: In the previous version, window positions were lost after switching views. Fix state persistence.
During iteration, avoid vague comments such as “the result is not good enough.” Instead, identify verifiable failure points, such as “the window header disappears after dragging to the right boundary” or “automatic research still enters items into the queue when resources are insufficient.” This makes correction more efficient and also enables systematic tracking of how the model responds to feedback.
5. Scoring Dimensions and Quantitative Evaluation
A weighted scoring system can help avoid relying on a single subjective judgment. An example rubric is:
{"functional_correctness": 0.30,"interaction_completeness": 0.20,"state_consistency": 0.15,"visual_consistency": 0.10,"code_maintainability": 0.10,"performance_stability": 0.10,"security_boundary": 0.05}
- Functional correctness measures whether the requested capabilities are implemented.
- Interaction completeness measures whether users can complete the intended operation flows.
- State consistency measures whether shared state remains correct across modules.
- Visual consistency measures whether the visual style remains coherent.
- Code maintainability measures modularity, naming, and overall organization.
- Performance stability measures whether the application noticeably degrades under high activity or extended runtime.
- Security boundary measures whether the generated result introduces unnecessary remote dependencies or unsafe execution.
Response speed is affected by the trial channel, concurrency and queueing, and resource scheduling. It should not be treated as a direct measure of model capability. Evaluation should record queue time, generation time, and retry count, then report capability scores separately from service experience.
6. Code-Level Acceptance Examples
For window dragging, the evaluation can require clearly structured core logic with traceable state:
const desktop = {windows: [],activeId: null};function activateWindow(id) {desktop.activeId = id;desktop.windows.forEach(w => {w.zIndex = w.id === id ? 100 : w.zIndex;});}function onPointerMove(win, event, bounds) {const nextX = event.clientX - win.offsetX;const nextY = event.clientY - win.offsetY;win.x = Math.max(0, Math.min(nextX, bounds.width - win.width));win.y = Math.max(0, Math.min(nextY, bounds.height - win.height));win.el.style.transform = `translate(${win.x}px, ${win.y}px)`;}
Automatic research can be evaluated with pseudocode such as:
function tickResearch(state) {if (!state.aiAssistant) return;const candidates = state.techTree.filter(t => !t.done && !t.queued).filter(t => state.resources >= t.cost).sort((a, b) => a.priority - b.priority);if (candidates.length > 0) {state.techQueue.push(candidates[0].id);candidates[0].queued = true;}}
If the generated code lacks resource checks, duplicate-queue protection, or queue feedback, it should lose points in the corresponding dimensions.
7. Engineering and Deployment Recommendations
In a production team, the evaluation service can be deployed in a general cloud environment or on infrastructure such as Baidu AI Cloud. Each generation task should run in an isolated container, with the prompt version, generated artifacts, runtime logs, and acceptance results recorded.
Automated tests can cover button clicks, drag boundaries, state recovery, and tick progression. Human review can focus on visual consistency and interaction feel. All results should be compiled into versioned reports so that a single trial does not replace systematic evaluation.
8. Conclusion
The value of evaluating complex interactive generation does not lie in producing impressive screenshots. It lies in using verifiable tasks to reveal a model’s actual ability to abstract requirements, manage state, schedule real-time behavior, and organize software.
With layered use cases, structured prompts, quantitative rubrics, and code-level acceptance testing, teams can make more objective decisions about whether a model is suitable for real development workflows and can design more reliable human–AI collaboration processes accordingly.
