Evaluating New-Generation LLM Code Generation with Classic Mini-Games: A Methodology for Frontend and Agent Tasks

    This article explores how classic mini-games can serve as a structured benchmark for LLM code generation, covering visual styling, logical modeling, physical plausibility, interaction completeness, and reinforcement-learning convergence. Drawing on hands-on tests of a new-generation model, it examines recurring failure modes such as chain-of-thought hallucination, missing real-world common-sense constraints, and unstable multi-round performance. It then proposes a reproducible, layered evaluation workflow that helps developers and enterprises assess frontend and Agent tasks and make better-informed model-selection decisions.

    In recent years, general large language models have continued to improve in code generation. “Generate a runnable mini-game from a single prompt” has become a common benchmark task for evaluating a model’s frontend coding and reasoning abilities. However, there is often a gap between demo videos and real-world usage. Based on hands-on testing of a new-generation model, this article distills a structured evaluation perspective for developers and enterprises to reference when selecting models.

    1. Why Use Mini-Games as a Test Benchmark

    Games that look simple as demos are actually highly comprehensive test scenarios, covering multiple capability dimensions at once:

    • Visual and styling ability: particle effects, lighting and shadows, and interface layout test the model’s command of modern CSS and graphics libraries.
    • Logical modeling ability: classic problems such as Klotski-style sliding block puzzles require correct data structures and state-transition design.
    • Physics and common-sense constraints: bowling simulations need to match real-world spatial scales, testing whether the model applies common-sense validation.
    • Interaction completeness: the generated page must not only “look right” but also “be playable”; interactivity is a hard requirement.
    • Algorithm and training ability: maze Q-learning requires the model to correctly implement the reinforcement-learning workflow and ensure that training converges.

    A single generation task can be passed by chance, but a multi-dimensional combination test more effectively exposes the boundaries of a model’s capabilities.

    2. Analysis of Typical Failure Points

    1. Visual Presentation: Acceptable Form, Questionable Details

    Most mainstream models generally produce acceptable page aesthetics today. Decorative elements such as particles and lighting can be generated in a plausible-looking way. The problem at the visual layer is an inconsistency between expression and implementation: a model may claim in its chain of thought that it will implement a certain visual style, such as an ink-wash texture, while the final output does not reflect that style at all. This phenomenon is essentially chain-of-thought hallucination: the reasoning text describes an effect that the model cannot actually deliver. During evaluation, claimed capabilities and delivered capabilities should be scored separately.

    2. Missing Common-Sense Constraints: Correct Code Does Not Equal a Correct Experience

    The bowling test exposed a subtler problem: the code was syntactically correct and the three.js calls were accurate, but the spacing between pins was too large, making a strike physically impossible. This shows that the model lacked common-sense modeling of real-world scale. Similar problems include insufficient contrast that makes an interface nearly unreadable: it works technically, but its usability is zero.

    One way to improve this is to introduce a common-sense validation checklist, for example:

    1. Physical-task checks:
    2. - Do key dimensions match real-world proportions?
    3. - Is the interaction goal (e.g., knocking down all pins in one shot) physically attainable?
    4. - Does the default color scheme meet basic readability (contrast thresholds)?

    3. Multi-Round Stability on Complex Tasks: One Success Does Not Prove Reliability

    Maze reinforcement learning was the case that best differentiated models in this round of testing. When the same task was repeated multiple times, model performance varied greatly:

    • Round 1: the agent’s size was out of control, occupying most of the maze.
    • Round 2: the maze itself was not generated correctly.
    • Round 3: the structure was complete, but the interaction broke and the result could not run.
    • Round 4: the coordinates of the agent behavior layer and the terrain layer were not aligned.

    Notably, despite multiple failures across rounds, the model performed strongly on Q-value visualization (annotating Q-values on floor tiles) and training convergence. Convergence is a metric that many comparable models struggle to achieve. This suggests that evaluation should use layered scoring: architecture correctness, rendering alignment, and training convergence are three independent dimensions to examine.

    3. Building a Reproducible Evaluation Workflow

    Based on the tests above, the following evaluation framework is recommended:

    1. Use a fixed task set: select standard tasks covering the five dimensions of styling, logic, physics, interaction, and algorithms.
    2. Sample repeatedly: repeat the same task at least 3 to 4 times and calculate the success rate rather than recording the best single result.
    3. Use double-blind scoring: remove model identifiers from generated results, then score them on four dimensions: runnability, visual quality, logical correctness, and common-sense plausibility.
    4. Distinguish “runs” from “usable”: track interaction completeness and real usability separately to avoid overestimating capability.
    5. Audit chain-of-thought consistency: check whether claims in the reasoning text are fulfilled in the final artifact and identify the hallucination rate.

    4. Conclusion

    The overall impression from hands-on testing can be summarized as the gap between expectations and reality. The new model showed progress in visual presentation, state modeling, and training convergence, but it still has clear weaknesses in common-sense constraints, inter-layer alignment, and multi-round stability. For developers, rather than focusing on launch-event demos, it is more useful to establish a fixed evaluation workflow with multiple tasks and repeated sampling, and to use quantifiable layered scores to guide technology selection. This comes closer to the model’s true capability level than any single hands-on experience.