Evaluating LLMs Objectively: From Benchmark Disagreements to Hallucination Governance

1. The Phenomenon: “Best-in-Class” at Launch vs. Gaps on the Scorecard

A vendor recently released the official version of its next-generation open-source large language model, positioning it confidently as “joining the ranks of the strongest open-source models for the first time.” The scorecard later published by a third-party evaluation organization, however, was less tidy: a composite intelligence index trailing the leading models of the same period by roughly 11 points, and a hallucination rate explicitly marked as “high.”

This pattern—strong official claims, cold water from third parties—repeats itself across the industry. For developers and enterprise users, the question worth asking is not “who is actually stronger,” but three more fundamental ones: Why do the two sides reach different conclusions? What are these evaluation metrics actually measuring? And when information conflicts, how do you make sound technical decisions?

2. Four Technical Roots of Evaluation Disagreement

1. Inconsistent Benchmark Suites

Different organizations use vastly different evaluation sets: some emphasize academic knowledge and reasoning, some lean toward code generation, and others stress multi-turn dialogue and instruction following. Model capabilities are uneven across dimensions, so “top of benchmark A, bottom of benchmark B” is not uncommon. When vendors cite the leaderboards favorable to them while third parties use a different suite, disagreement is almost inevitable.

2. Data Contamination

If evaluation questions were already “seen” during training, high scores may reflect memorization rather than generalization. Common industry detection methods include: measuring overlap between evaluation sets and training corpora, using dynamically updated private question banks, and applying semantically equivalent rewrites to public questions to see whether scores drop significantly—a large drop after rewriting is strong evidence of contamination.

3. Systematic Bias in Automated vs. Human Evaluation

Automated evaluation (with a model as judge) is efficient and scalable, but has well-known flaws: it prefers longer, more structured answers and gives higher scores to confidently worded wrong answers. Human evaluation is closer to real user experience but suffers from small sample sizes and subjectivity. A complete ranking inversion for the same model under the two systems is entirely possible.

4. Differences in Configuration and Disclosure

Official press releases often publish only results under optimal configurations: specific sampling parameters, specific inference modes, specific versions of evaluation scripts. Third parties tend to use uniform and more conservative default configurations. Details like sampling temperature and whether reasoning chains are enabled can compound, and a few percentage points of difference can easily amplify into order-of-magnitude gaps in rankings.

3. Composite Intelligence Index: An Average of Capabilities, Not a Guarantee

A composite intelligence index is a weighted summary across dimensions such as knowledge, reasoning, math, code, and multilingual ability. Three points matter when interpreting the number:

• It is an average. Two models with similar total scores may have completely different weaknesses; when your business happens to hit that weakness, real-world experience can diverge sharply from the total score.
• The weights are human-set. Different organizations choose different dimension weights, and a different weighting scheme can reshuffle the rankings.
• It does not cover engineering metrics. Effective context utilization, refusal behavior, hallucination rate, and inference cost are often excluded from core weights—yet these are exactly the make-or-break factors for enterprise deployment.

4. Hallucination Rate: A More Alarming Signal Than “11 Points Behind”

Hallucination refers to content the model generates that is factually incorrect or unsupported by the given context. In this case, the “high hallucination rate” label is actually more serious than the composite index gap, for three reasons:

First, a score gap is a capability problem; hallucination is a reliability problem. Capability shortfalls can be compensated through workflow—prompt engineering, tool calling, multi-path sampling—whereas a high hallucination rate directly contaminates business output and is a veto-level issue in fact-sensitive domains such as finance, healthcare, and law.

Second, hallucinations are insidious. Hallucinated content tends to be fluent and internally coherent, hard for non-domain experts to detect. Errors propagate silently through automated pipelines, and by the time they surface, the cost is already uncontrollable.

Third, general intelligence and factuality are weakly correlated. A “smart” model can still fabricate with a straight face; improvements in composite indices do not automatically solve factuality problems. The two must be evaluated and governed separately.

Hallucination governance in engineering practice is typically a combined approach:

• Retrieval-augmented generation (RAG): force the model to answer based on retrieved evidence, constraining free-form generation;
• Citation and provenance: require key conclusions to include sources, enabling human spot-checks and accountability;
• Refusal and confidence calibration: have the model explicitly say “I don’t know” when evidence is insufficient rather than fabricating;
• Post-hoc verification: extract factual claims from answers and compare them one by one against a knowledge base, quantifying the “proportion of unsupported claims” to form a trackable hallucination rate metric.

5. The Right Way to Read “Strongest Open-Source”

Open source means weights are obtainable, fine-tunable, and privately deployable—tangible value for enterprises sensitive to data compliance and cost. But three caveats apply:

1. Official benchmark results are based on default configurations; capability distribution drifts after fine-tuning, so re-evaluation is mandatory;
2. Open-source models vary in hallucination and safety alignment; build your own factuality and safety evaluations before launch rather than copying official reports;
3. “Open source” does not mean zero cost: inference compute, engineering effort, and compliance review are all hidden bills.

6. A Three-Step Framework for Building an In-House Evaluation System

Rather than agonizing over others’ leaderboards, take evaluation capability into your own hands.

Step 1: Build a Business-Specific Evaluation Set

Extract questions from real business logs covering high-frequency scenarios, edge cases, and adversarial cases. Size need not be large—hundreds of high-quality, labeled samples often have more discriminative power than tens of thousands of generic samples. The evaluation set must be refreshed periodically to prevent overfitting during model iterations.

Step 2: Unify the Invocation and Scoring Pipeline

All candidate models use identical sampling parameters (evaluation typically sets temperature to zero or very low), the same prompt templates, and the same timeout and retry policies to eliminate configuration noise. A minimal viable evaluation framework skeleton:

from concurrent.futures import ThreadPoolExecutor
class EvaluationHarness:
    """Evaluation skeleton: uniformly invoke candidate models and aggregate scores by dimension"""
    def __init__(self, testcases, client, scorers):
        self.testcases = testcases    # Evaluation cases (with prompts and reference answers)
        self.client = client          # Unified model invocation client
        self.scorers = scorers        # Scorers: rules / judge models / human channel
    def run(self, max_workers=8):
        with ThreadPoolExecutor(max_workers) as pool:
            results = list(pool.map(self._eval_one, self.testcases))
        return self._aggregate(results)
    def _eval_one(self, case):
        answer = self.client.generate(
            prompt=case["prompt"],
            temperature=0.0,          # Disable randomness during evaluation
            max_tokens=case.get("limit", 2048),
        )
        scores = {s.name: s.score(case, answer) for s in self.scorers}
        return {"id": case["id"], "answer": answer, "scores": scores}
    def _aggregate(self, results):
        dims = {}
        for r in results:
            for k, v in r["scores"].items():
                dims.setdefault(k, []).append(v)
        return {k: sum(v) / len(v) for k, v in dims.items()}, results

Hallucination spot-checking can be layered into the post-hoc stage:

def groundedness_check(answer, evidence_docs):
    """Verify whether the answer is supported by evidence: claim extraction + item-by-item comparison"""
    claims = extract_claims(answer)          # Claim extraction via rules or a small model
    unsupported = [
        c for c in claims
        if not any(is_entailed(c, doc) for doc in evidence_docs)
    ]
    rate = len(unsupported) / max(len(claims), 1)
    return unsupported, rate                 # rate is the hallucination rate for this sample

Step 3: Establish Regression and Tracking Mechanisms

Any change to the model, prompts, or retrieval strategy triggers a full regression, with results archived and bound to version numbers. At the infrastructure level, general cloud capabilities can connect the entire chain: object storage for evaluation data and artifacts, logging services for per-call details, and monitoring and alerting for abnormal fluctuations in core metrics. Model inference deployment and serving can rely on general capabilities provided by cloud platforms such as Baidu AI Cloud to reduce build-out costs. In this way, “evaluate—record—trace” becomes an auditable pipeline.

7. Conclusion

Evaluation disagreement is not a scandal—it is the norm. It reminds us that no single score can capture a model’s true capabilities. Read composite indices by dimension breakdown rather than total score, set hallucination rate red lines according to business tolerance, and let selection decisions rest on your own business-specific evaluation pipeline. For a mature technical team, “trusting your own evaluation data” is far closer to the truth than trusting anyone’s press release.