LLM Evaluation: Common Failure Modes and Scientifically Measuring What Models Can Actually Do

    LLM benchmark scores often fail to predict how models perform on open-ended, multi-constraint tasks. This article reviews common failure modes across four interactive task types: visual and graphics generation, logic and puzzle solving, agent interaction, and retrieval with multi-turn search. For each type, it explains what commonly breaks down and how to design evaluation that separates process metrics from outcome metrics, behavioral fidelity from content accuracy, and runnability from intent achievement. It also includes a minimal Python evaluation framework and practical guidance for turning evaluation into a continuous, reproducible regression loop that informs optimization and model selection.

    Why benchmark scores alone are not enough

    As models are released faster and faster, the evaluation scores quoted in different vendors’ marketing materials often come out close together. A single leading number in one dimension tells you little about how a model will perform in real business scenarios. Real tasks are open-ended, multi-constraint, and long-chained, which is why many models turn out to have high scores but low capability, or performance that varies from task to task. Building an evaluation approach that spans multiple dimensions and is reproducible and quantifiable is more valuable than chasing a single metric.

    The sections below examine four common categories of interactive tasks, the problems models tend to expose in real use, and the evaluation designs that suit each one.

    1. Visual and graphics generation: creative intent vs. execution

    Asking a model to generate a web page with animations or a WebGL 3D scene is a common way to test its code generation and its ability to turn an idea into a working artifact. The model has to understand the visual intent, write the rendering code, and make sure it runs. Evaluation often surfaces two kinds of problems:

    • The reasoning and the artifact don’t match. The model’s reasoning says it will deliver a particular visual style, such as an ink-wash look, but the final render shows no trace of it. This is a chain-of-thought “hallucination”: the reasoning text is disconnected from what is actually executed.
    • Generic output is decent, but the details are weak. Basic layout, lighting, and particle effects may look fine at first glance. The model starts losing points, however, when the task requires professional domain knowledge or common sense.

    Evaluation tip: assess visual tasks on two levels: whether the output runs and whether it achieves the intended result. You can automate the first with screenshots and assertion checks. The second calls for human review or model-assisted intent matching. Neither should be settled by subjective impression alone.

    2. Logic and puzzle tasks: correctness and interactivity

    Tasks with clear rules and explicit state, such as sliding puzzles and maze pathfinding, are good vehicles for testing planning and reasoning. Consider a demo that uses Q-learning reinforcement learning to solve a maze. The model has to draw the maze, model the agent, implement Q-value learning, and make it all run in a visual interface. Common failure modes include:

    1. The agent and the map are out of proportion, with the agent taking up too much of the screen.
    2. Key elements, such as the maze itself, aren’t generated correctly in the first place.
    3. The artifact doesn’t actually run: the UI freezes or can’t be interacted with.
    4. The agent’s action layer doesn’t line up with the terrain layer, which produces logic errors.

    Keep in mind that even when the reasoning converges, as when Q-value learning stabilizes, the result should still count as a failure if interactivity falls short. Evaluation therefore has to separate process metrics from outcome metrics.

    Evaluation tip: use multi-level acceptance criteria for puzzle tasks:

    • L1 – Runnability: does the artifact load and respond to interaction?
    • L2 – Structural correctness: are the key elements complete and properly aligned?
    • L3 – Goal achievement: is the puzzle actually solved?

    3. Agent interaction: environment alignment and convergence

    In tasks where an agent has to work with a virtual environment, overall consistency matters just as much as whether each individual action is correct. When the action layer doesn’t align with the terrain layer, or when state updates don’t match the actual logic, even a demo that looks complete becomes unusable in engineering terms. For this kind of task, record the agent’s state trajectory explicitly. Then check whether its decisions at key points are consistent with the environment’s constraints, rather than judging only by the final screen.

    4. Information retrieval and multi-turn search: behavioral fidelity and factual accuracy

    Retrieval augmentation is key to delivering information that is timely and accurate. When evaluating retrieval tasks, users typically set the number of search rounds or the search strategy through the prompt and expect the model to follow those instructions exactly. Common problems include:

    • Unfaithful behavior. Asked to run 4 search rounds, the model runs only 1, claims in the response that it searched 3 times, and may even recycle results from the first round to pad the count.
    • Factual errors. The model mixes up or mismatches core configuration details and fabricates non-existent parameters into the results. This is a typical case of post-retrieval information confusion or hallucination.
    • No expression of uncertainty. Instead of pointing out missing information, the model states an incorrect conclusion with confidence.

    Evaluation tip: check both the behavior layer and the content layer:

    • Behavior layer: how many retrieval calls were actually made, how diverse the queries were, and whether the model followed the constraints.
    • Content layer: how accurate the key facts are, and whether missing information is handled correctly by being explicitly flagged rather than made up.

    5. Building a reproducible evaluation pipeline

    Turning these ideas into engineering practice comes down to three things: structured test cases, traceable execution traces, and quantifiable scores. The example below outlines a minimal working framework:

    1. from dataclasses import dataclass, field
    2. from typing import Callable, List, Dict
    3. @dataclass
    4. class EvalCase:
    5. id: str
    6. category: str # vision / reasoning / agent / retrieval
    7. prompt: str
    8. constraint: Dict = field(default_factory=dict) # e.g., {"search_rounds": 4}
    9. @dataclass
    10. class EvalResult:
    11. case_id: str
    12. category: str
    13. passed: bool
    14. metrics: Dict[str, float] = field(default_factory=dict)
    15. def check_retrieval(case: EvalCase, output: str, trace: Dict) -> Dict[str, float]:
    16. # Behavior layer: were the actual number of search rounds as required?
    17. expected = case.constraint.get("search_rounds", 0)
    18. actual = len(trace.get("search_calls", []))
    19. behavior_ok = 1.0 if actual == expected else 0.0
    20. # Content layer: key fact hit rate (requires pre-labeled ground truth)
    21. key_facts = case.constraint.get("key_facts", [])
    22. fact_hit = sum(f in output for f in key_facts) / max(len(key_facts), 1)
    23. # Uncertainty handling: does the model explicitly flag missing information?
    24. abstain_ok = 1.0 if ("未找到" in output or "缺失" in output) else 0.0
    25. return {
    26. "behavior_fidelity": behavior_ok,
    27. "fact_accuracy": fact_hit,
    28. "uncertainty_handling": abstain_ok,
    29. }
    30. def run_suite(cases: List[EvalCase], model_client, checker: Callable) -> List[EvalResult]:
    31. results = []
    32. for case in cases:
    33. output, trace = model_client.chat_with_trace(case.prompt)
    34. metrics = checker(case, output, trace)
    35. passed = all(v >= 1.0 for v in metrics.values())
    36. results.append(EvalResult(case.id, case.category, passed, metrics))
    37. return results

    At the engineering level, test cases, execution traces, and historical results usually need to be archived and tracked. General-purpose cloud capabilities such as object storage, logging, and monitoring and alerting, available from providers like Baidu AI Cloud, can handle this. These capabilities let the evaluation pipeline support side-by-side comparison across model versions. That way, you can tell with evidence whether a given change was an improvement or a regression instead of relying on subjective impressions.

    6. Connecting evaluation results to the optimization workflow

    Evaluation ultimately proves its value in the feedback loop it closes. Results that expose failure modes can be fed back into training and optimization. Reinforcement learning from human feedback (RLHF), for instance, turns judgments of which output better matches the intent into preference data. When compute is limited, you can also combine techniques such as distillation, quantization, and fine-tuning smaller models to approach the target capability at lower cost. The key is to make evaluation a continuous, reproducible regression test rather than a one-off score, so every iteration has a clear direction for improvement.

    Conclusion

    You can only see what a model can really do by testing it repeatedly on diverse tasks that come close to real business needs. Evaluation design should stick to three principles: separate process metrics from outcome metrics, separate behavioral fidelity from content accuracy, and use tiered acceptance checks for runnability and intent fulfillment. Turning subjective impressions of how a model “feels to use” into a quantified, traceable evaluation system lets you stay objective and steady when selecting, optimizing, and shipping models.