Why Aggregate Model Scores Can Mislead: Capability Breakdowns and Hallucination Control

    Aggregate benchmark scores compress many weighted subtasks and often diverge from hands-on experience in coding and agent work. This article explains that gap, then covers overconfident answers and hallucination in closed-world knowledge QA, weaknesses in long-document and cross-page table handling, the difficulty of professional reasoning, why programming and agent tasks can look strong, and how to choose models with a business-aligned evaluation loop.

    1. Why aggregate scores and subjective experience diverge

    Public aggregate scores are usually weighted sums of many subtasks. Weights, task sampling, prompt design, and scoring rules change as benchmarks iterate. An aggregate score is a compressed metric: useful for coarse screening, but a poor direct map to any single business capability. In programming and agent tasks, developers experience models that work with clear context, iteration, and verification. An aggregate score may also include a large share of closed-world knowledge QA, long-document information extraction, and professional reasoning. Different task mixes naturally produce a gap between rankings and subjective experience.

    Item-level scores are more diagnostic than the total. Failure cost matters especially: in knowledge QA, a wrong answer can be worse than an explicit abstention; in long-document tasks, missing a key field can be more serious than imperfect formatting; in professional reasoning, a wrong step can be riskier than refusing to answer. Model selection should weigh capability breakdowns, the distribution of business tasks, and the cost of failure together—not a single aggregate number.

    2. A core drag on the score: overconfident answers and hallucination in open-world knowledge QA

    Closed-world knowledge QA typically forbids web search and tool use. The model must rely on parametric memory for accurate names, dates, or numbers. A common scoring design awards points for correct answers, deducts points for wrong answers, and gives zero for an explicit “I don’t know.” That rule treats abstention as an acceptable strategy: a system that only abstains does not go negative, but it also earns no positive score; answering too often can accumulate penalties from errors.

    Limited knowledge capacity and long-tail coverage are normal for smaller models, so knowledge gaps are expected. The real problem is poor confidence calibration: the model still produces fluent answers on low-confidence questions—overconfident answering. Control can be approached on four levels:

    1. Confidence calibration. Use a validation set to relate model confidence to actual accuracy, and set an abstention threshold. Answer high-confidence questions; refuse or hand off low-confidence ones.
    2. Training and prompt constraints. Add examples and preferences for “refuse when evidence is insufficient,” so the model can distinguish “known,” “uncertain,” and “unknown.”
    3. Decoding strategy. Reduce random overconfident answers with multi-sample consistency, temperature constraints, and candidate scoring.
    4. Production-scenario enhancement. Where retrieval is allowed, add retrieval augmentation and fact checking. In closed evaluations that forbid network access, treat this as a boundary of parametric knowledge. Engineering workarounds should not blur the evaluation definition.

    Example pseudocode:

    1. def answer_with_calibration(question, model, threshold=0.72):
    2. candidates = model.generate(question, n=3, temperature=0.2)
    3. answer, confidence = aggregate_and_score(candidates)
    4. if confidence < threshold:
    5. return "I don't know"
    6. if not evidence_check(answer):
    7. return "I don't know"
    8. return answer

    The threshold is not a fixed parameter. It must be tuned on a validation set. Monitor refusal rate, error rate, and task completion rate together so hallucination is reduced without excessive refusal.

    3. Shortfalls in long documents and cross-page tables

    Long-document tasks are not simply “make the context longer.” They usually require layout parsing, table structure recovery, cross-page entity alignment, evidence localization, and structured output. Common failures include slices that split tables, interference from headers and footers, lost cross-page headers, inconsistent numeric units, and misaligned citations.

    Useful directions include layout-aware parsing that keeps page numbers and coordinate metadata; rebuilding chunks by section and table; two-stage retrieval with coarse recall plus fine ranking; a second check on critical numbers; structured schemas that constrain output; and an evidence chain for traceability. Evaluation should track field-level accuracy, evidence recall, cross-page alignment, and end-to-end completion—not only whether the final text reads smoothly.

    4. Why professional reasoning is hard

    Professional reasoning requires domain knowledge, symbolic computation, units and dimensions, approximation conditions, and multi-step reasoning. Smaller models often lack knowledge density and stable reasoning chains. They skip intermediate steps, omit conditions under which a formula applies, or convert units incorrectly. Improvements can include a domain knowledge base, formula and unit checkers, subproblem decomposition, self-consistency voting, and failure fallback. If the task allows tools, a calculator or similar tool can reduce numeric errors. Evaluation should cover step correctness, final-answer correctness, and whether refusals are reasonable.

    5. Why programming and agent tasks can look strong

    Programming and agent tasks often have clear context, executable checks, and multi-turn iteration. The model can revise code from test feedback and use tools for retrieval, file handling, and task orchestration. These tasks match day-to-day development, so developers readily form the impression that the model “works well.”

    A strong score on one slice does not mean strength on every task. Distinguish task type, tool availability, the execution sandbox, and failure cost. In engineering, use a tool protocol, an execution sandbox, test-driven iteration, log replay, and permission control so agent work stays observable and reversible.

    6. Sound model selection and an evaluation loop

    Do not select a model from a single aggregate score. Build an internal gold set that covers real business tasks; read performance by capability; set a reasonable abstention policy; run A/B comparisons; and keep monitoring task completion rate, first-turn accuracy, hallucination rate, refusal rate, average turns, evidence recall, long-document field accuracy, and step correctness in professional reasoning.

    On the evaluation stack, general-purpose container platforms, object storage, logging, and monitoring and alerting—such as the corresponding categories offered by Baidu AI Cloud—can support an observable evaluation loop. The concrete architecture should follow the team’s existing stack. The principle is simple: use the aggregate score for trends, item scores for weaknesses, the business loop for fit, and failure cost to set the abstention policy.