Multimodal LLM Hands-On Test: Style Consistency, Physical Scale, and State Instability

    This article reports four hands-on tests of a Chinese large model's multimodal agent capabilities. The tests cover visual generation, spatial layout, physical scale, and reinforcement learning coding. The model performs acceptably in visual aesthetics and algorithm understanding. Style consistency, physical scale, and front-end state management are its main weak points. The tests are limited in scope and are not an official evaluation.

    Multimodal large language models have improved noticeably over the past year, but a gap remains between recognizing image content and understanding the rules of the physical world. The vendor of a Chinese large model’s latest release emphasizes its multimodal agent capabilities and claims it can autonomously perceive, plan, and execute tasks in interactive settings. This article tests the model with four sets of hands-on experiments covering visual generation, spatial arrangement, physical scale, and reinforcement learning programming. The sample is limited, and these are not official evaluations. The conclusions apply only to the prompts and environments described here.

    In short, the model performs acceptably in visual aesthetics and algorithm understanding. Style consistency, physical scale, and front-end state management are its main points of instability.

    1. Visual Generation: Reasoning Commitments and Actual Output Diverge

    By default, the model produces good lighting and image layering. The problem appears in style control. In one task, the model’s reasoning explicitly planned an “ink wash” style, but the final image looked more like an antique wood-grain texture and did not reflect ink wash at all.

    Several causes could explain this. One hypothesis is information loss during handoff between modules: the abstract language description may be compressed into low-dimensional features before it reaches the visual generator. Another hypothesis is that the prompt was ambiguous, or that the generator’s training data covers “ink wash” poorly. The current tests cannot distinguish between these possibilities, because the intermediate text representation was not recorded separately.

    The verification method is as follows. Fix the prompt and random seed, then swap the visual generator as a control. Record the model’s intermediate planning text. Compare output distributions with and without a style reference image. Only after these ablation experiments can the layer where the problem occurs be identified.

    For developers, do not leave style instructions to be passed implicitly inside the model. Add an external style control layer instead, for example by supplying a reference image, or by using a style embedding vector to constrain the distribution of generated results.

    2. Spatial Arrangement and Scale: Correct Placement Does Not Guarantee Accurate Scale

    In the Huarong Dao sliding puzzle test, every block reached its target position, and no relative positions were scrambled. The model performed consistently on two-dimensional arrangement. However, its reasoning also proposed an additional “ink wash effect” goal, and the final result did not show it.

    The bowling scene exposes scale problems more clearly. The test environment was built with Three.js. The model generated a complete set of pins, but the spacing between them was too large. Under common bowling specifications, the standard center-to-center distance between pins is 30.48 cm (12 inches), and the maximum pin body diameter is about 12.07 cm (4.75 inches). The center spacing is therefore about 2.5 times the pin diameter. Some sources cite 34.2 cm, but that is not a standard value and should not be used as a reference.

    The deviation may also come from the test environment. Three.js unit settings, scene scaling, or numeric conversion of the model’s output could each affect the result. This observation therefore shows only that, under this prompt and rendering setup, the output scale did not match standard specifications.

    In engineering practice, physical scenes generated by an LLM need scale validation. A lightweight approach is a rule-based check, such as verifying that the distance between adjacent entities falls within a defined range. A more costly but more reliable approach is to connect a physics engine simulation to check collisions and spacing. The choice depends on precision requirements and development budget. Either way, define units explicitly in the code and confirm the coordinate system.

    3. Reinforcement Learning Maze: Four Interactions, Four Failures

    The third test asked the model to draw a maze with HTML/Canvas, implement a Q-learning pathfinding agent, and keep the rendering layer and the logic layer pixel-aligned. This was the most difficult task in the test. It covered code generation, algorithm understanding, front-end rendering, and coordinate handling. The tester gave the model several rounds of correction, but the model never fully passed.

    The four failures were as follows:

    • Round 1: The agent was rendered too large and covered about half of the maze. The layout calculation may have set the size incorrectly.
    • Round 2: The maze was drawn, but the agent did not appear on the canvas, and interaction could not be triggered.
    • Round 3: The maze and the agent were both rendered, but clicks and keyboard input had no effect. The cause may lie in event listeners or state updates.
    • Round 4: The agent’s pixel positions were systematically offset from the logical grid coordinates. Its movement path did not line up with the passages. This may be a coordinate transformation error.

    The four failures have different causes. Some have attributed them to a single root cause, a breakdown in cross-language state management, but this remains an unverified hypothesis. Testing it would require logging each round’s model version, temperature, prompt, canvas size, Q-learning hyperparameters, and the criteria used to judge success.

    The test also showed real strengths. The model overlaid Q-values as a heatmap on the floor tiles, and during training, the Q-values gradually converged. This suggests a reasonable grasp of the reinforcement learning workflow. Its weak points are front-end engineering and unified management of coordinates.

    4. Three Principles for Developers

    Based on these results, we recommend three principles for multimodal agent projects.

    First, decouple the layers. Do not rely on the model end to end to guarantee physical correctness and style consistency. Set up independent validation steps between language planning, visual rendering, and business logic.

    Second, externalize state. Store key state such as coordinates, dimensions, and logical identifiers in external data structures. Have the code read and write this state in one place, rather than relying on the model to maintain it across multiple functions. Use the following checklist:

    • Define the coordinate system and units in the documentation.
    • Keep a single external store for each key piece of state.
    • After each interaction, check the difference between the rendered position and the logical position.

    Third, rehearse in a sandbox. Run generated code in a low-cost environment before release, especially for physics engines, Canvas rendering, and reinforcement learning training. Small-scale rehearsal exposes size, coordinate, and event-binding problems early and shortens debugging time.

    5. Guidance for General Users

    Model marketing often showcases high scores on single tasks. Real multimodal interaction is more complex than benchmark tests. The failure cases in this article document the model’s current capability limits rather than rejecting it outright. When choosing a model, test it on your own real tasks at small scale instead of relying only on demos.

    Conclusion

    This article covers only four limited scenarios. The results depend on the prompts, the environment, and the model version. They do not represent the model’s overall performance, and this is not an official evaluation. In these samples, the model shows usable visual aesthetics and reinforcement learning algorithm understanding. Style consistency, physical scale, and front-end state management remain its main weak points. For developers, failure cases often provide more troubleshooting clues than a single polished demo.