How to Scientifically Evaluate the Real Capability Boundaries of Large Language Models

    A practical framework for evaluating large language models that goes beyond first-glance impressions. It breaks evaluation down into six dimensions—frontend and visualization code generation, physical common sense and logical consistency, agent task completeness and convergence, multi-turn retrieval faithfulness, chain-of-thought hallucination, and factual accuracy—and shows how to turn vague subjective judgments into verifiable criteria through structured, reproducible test cases and automated scoring. It also covers engineering the evaluation pipeline with general cloud capabilities for large-scale, repeatable regression testing, so teams can make evidence-based model selection decisions.

    1. Starting Point: From “Usable” to “Verifiable”

    Whenever a new general-purpose large model is released, developer communities go through a wave of trials. But a common mistake is equating “impressive at first glance” with “meeting the bar.” Descriptions like “the rendering looks slick,” “the lighting and particles look convincing,” or “the interface is kind of cool” can’t support any serious technical selection decision, nor do they form a comparable baseline across model iterations.

    Truly valuable evaluation turns vague subjective impressions into verifiable criteria, answering three progressively deeper questions:

    • Functionality: Does the task actually run end to end (does it execute, can you interact with it, can it be reproduced)?
    • Quality: Does the result conform to domain common sense and expectations?
    • Faithfulness: Does the model’s behavior match what it “claims” to do (especially for retrieval, reasoning chains, and multi-step tasks)?

    Around these three layers, I’ve distilled six evaluation dimensions from the scenarios that most reliably expose problems in real testing. I’ll cover each below.

    2. Dimension 1: Frontend and Visualization Code Generation

    Generating runnable web pages, animations, or 3D scenes is a high-frequency task for testing a model’s ability to go “from natural language to executable code.” It’s worth looking at this in two steps:

    1. Visual layer: Are the lighting, particles, composition, and materials coherent? Most models now clear this “passing line,” so it’s a baseline capability and shouldn’t be a discriminating metric.
    2. Logic correctness layer: Are the interaction logic, physics rules, and state management correct? This layer is the real dividing line.

    The takeaway: visual polish does not equal logical correctness. A page with maxed-out particle effects and beautiful rendering is still a failed case if its interaction logic is wrong or it crashes on launch. So the evaluation script must include a hard pass/fail check: “does it run automatically and pass interaction assertions?”

    3. Dimension 2: Physical Common Sense and Logical Consistency

    There’s a particularly insidious class of errors: the code runs and the scene renders, but it violates basic common sense. For example, ask a model to generate a bowling scene. The scene does render, but the gaps between the pins are too large—there’s physically no way to knock them all down in a single roll.

    These problems can’t be judged by eyeballing “does it look right?” Instead, you need common-sense checks:

    • Spatial relationship checks: Do object spacing, counts, and arrangement match real-world constraints?
    • Physics rule checks: Do collisions, gravity, and bounces follow basic physics?
    • Business constraint checks: Does the scene satisfy the implicit premises given in the task?

    For quantifiable scenarios (counts, coordinates, spacing), you can embed assertable checkpoints in the generated code, turning “common sense” into “testable assertions.”

    4. Dimension 3: Completeness and Convergence of Agent Tasks

    Asking a model to build an agent with reinforcement learning capability (for example, pathfinding in a maze with Q-learning) is one of the toughest all-around evaluations, because it simultaneously tests algorithmic correctness, rendering completeness, and interactivity. Common failure modes observed in practice include:

    • First attempt: The agent’s size runs out of control, taking up nearly half the maze.
    • Second attempt: The maze terrain doesn’t render at all.
    • Third attempt: The maze is there, but the page isn’t interactive (it freezes, can’t play).
    • Fourth attempt: The agent’s action layer and the terrain layer aren’t aligned—logic and visuals are disconnected.

    One good practice worth praising: visualizing the Q-values as a “heatmap” on the floor, with the training process able to converge. Convergence is the key signal for whether a reinforcement learning implementation is correct; failure to converge usually means something is wrong with the algorithm or the state encoding.

    When evaluating this kind of task, I recommend judging in a strict sequence: “runs completely → layers aligned → interaction works → algorithm converges.” If any link breaks, mark that level as failed, and avoid misjudging a “partial success” as an “overall pass.”

    5. Dimension 4: Faithfulness of Multi-Turn Retrieval

    When a task requires retrieving up-to-date information, whether the model “does what it says” becomes especially important. A typical scenario: requiring the model to perform four rounds of search. The problems that showed up in practice are quite telling:

    • The model claims on the surface that it performed multiple retrievals, but actually only ran the first search, then repeatedly reused the first round’s results in the body text.
    • In other words, the model’s “process description” is inconsistent with its “actual behavior.” This is a faithfulness problem, and it’s more worrying than a simple wrong answer.

    By contrast, robust behavior is “controllable in count”: search exactly as many times as required, no more and no less, with each retrieval genuinely happening and influencing the conclusion.

    The key to evaluating retrieval faithfulness isn’t whether the final answer is “correct,” but whether the “claimed behavior” matches the “actual behavior in the logs.” You can compare them with a structured record like this:

    1. {
    2. "task": "multi_turn_search",
    3. "required_rounds": 4,
    4. "claimed_rounds": 4,
    5. "actual_rounds": 1,
    6. "result_reuse": true,
    7. "faithfulness_pass": false
    8. }

    6. Dimension 5: Detecting Chain-of-Thought Hallucination

    A model may “plan” certain effects in its reasoning chain (chain of thought) but never actually deliver them in the final output. This is a classic case of chain-of-thought hallucination. For example, the model states in its reasoning that it will “present an ink-wash style,” yet the generated image shows no ink-wash elements whatsoever.

    The danger here is that the reasoning chain reads as internally coherent and convincing, easily giving reviewers the false impression that “it really did it.” The countermeasure is to stick to the “the artifact is the ground truth” principle: claims made in the reasoning chain must be fulfilled by the final artifact; anything not fulfilled is judged as hallucination. During evaluation, you can align “planned items” against “implemented items” one by one:

    1. planned_items = ["ink-wash style", "negative-space composition", "light ink shading"]
    2. rendered_items = detect_visual_features(output_image) # Verified manually or by a vision model
    3. hallucination = [x for x in planned_items if x not in rendered_items]

    7. Dimension 6: Factual Accuracy and Configuration Details

    In retrieval tasks, the most dangerous error is fabricating core facts. For example, when querying a car model’s configuration, the model invents configuration items that don’t exist in the source (such as a suspension type it doesn’t actually have). This is a serious factual error—it shows up in critical decision-making information and can directly lead users to misjudge.

    A robust reference standard is this: faced with a configuration that doesn’t exist, the model shouldn’t fabricate it—instead it should proactively note that “this item is not equipped.” This tells us that when evaluating factual accuracy, you should specifically design “trap questions”:

    • Ask about a nonexistent attribute, and see whether the model fabricates an answer.
    • Ask about an easily confused detail, and see whether the model can distinguish it precisely.
    • Ask for a conclusion that requires cross-verification, and see whether the model stays faithful to the retrieval results.

    8. Evaluation Engineering: Building a Reproducible Evaluation Set

    For these six dimensions to be put into practice, they must be captured as reproducible, quantifiable evaluation assets. I recommend a unified structured case format to describe each test point:

    1. - id: maze_qlearning
    2. dimension: agent_task
    3. input: "Implement maze pathfinding with Q-learning and visualize it"
    4. checks:
    5. - type: runnable
    6. - type: layer_aligned
    7. - type: interactive
    8. - type: converged # Key: whether training converges
    9. weight: 0.3 # This dimension carries a high weight
    10. - id: search_faithful
    11. dimension: retrieval
    12. input: "Force 4 rounds of search and summarize"
    13. checks:
    14. - type: claimed_eq_actual # Claimed rounds == actual rounds
    15. - type: no_hallucination

    Paired with an automated scoring script, you can run tests in bulk and output a structured report:

    1. def evaluate(case, model_output, execution_log):
    2. results = {}
    3. for check in case["checks"]:
    4. t = check["type"]
    5. if t == "runnable":
    6. results[t] = can_run(model_output)
    7. elif t == "converged":
    8. results[t] = training_converged(execution_log)
    9. elif t == "claimed_eq_actual":
    10. results[t] = (execution_log["claimed_rounds"]
    11. == execution_log["actual_rounds"])
    12. # ... other check types follow the same pattern
    13. score = sum(case["weight"] * results.get(c["type"], False)
    14. for c in case["checks"])
    15. return score, results

    The core principle: every case must have clear, automatically determinable pass/fail conditions, avoiding vague scoring in the style of “it feels pretty good overall.”

    9. Evaluation Engineering in the Cloud

    When evaluation scales up and requires collaboration among multiple people or periodic regression testing, you can leverage general cloud capabilities to engineer the evaluation pipeline:

    • Object storage: Centrally store evaluation case sets, input prompts, and artifacts (generated pages, images, logs).
    • Container platform: Batch-launch the inference environments of the models under test in a containerized way, ensuring a consistent runtime and reproducible results.
    • Message queue: Decouple “case distribution” from “result collection,” supporting large-scale concurrent evaluation.
    • Logging and monitoring/alerting services: Capture the complete call chain and retrieval logs for each inference run, used for faithfulness verification, and set alerts for regression failure rates.

    This ensures the reproducibility of evaluation while making it easy to run quick regression comparisons as models iterate.

    10. Conclusion

    Evaluating a large model’s real capabilities is essentially a process of “de-mystification”: stripping away surface factors that easily win favor—visual splendor, flashy interfaces—and focusing on three hard standards: whether the function runs, whether the result matches common sense, and whether the process is faithful. The six dimensions—visualization generation, physical common sense, agent convergence, multi-turn retrieval faithfulness, chain-of-thought hallucination, and factual accuracy—form a fairly complete practical evaluation framework. By capturing them as a structured, automatically determinable evaluation set and engineering it with cloud capabilities, teams can make evidence-based technical selections in the face of a constant stream of new models, rather than being led around by “it looks pretty good.”