Why “Is It Good?” Is Hard to Answer

When people evaluate large language models, they usually rely on impressions from a handful of conversations: Is the response fluent? Can it understand complex questions? Was that one impressive success or glaring failure memorable? This kind of experience has some value, but it suffers from obvious sample bias. Users tend to remember striking answers and absurd mistakes, while the average accuracy across similar tasks goes unmeasured.

Another problem is that model capability varies widely by task. The same model may perform consistently on code completion, frequently miss details in long-document QA, and produce plausible but unverifiable answers for obscure factual queries. So before asking whether a model is “good,” you need to answer three questions: What task is it used for? What form does the input take? What counts as success?

Define Task Boundaries Before Testing

Before running any tests, break your business requirements down into quantifiable task descriptions that cover at least the following:

1. Task type: information extraction, summarization, code explanation, QA, multi-turn dialogue, or image-text understanding.
2. Input characteristics: text length, degree of structure, presence of tables or images, language.
3. Output constraints: whether the format is fixed, whether citations from the source are required, how much error is tolerable.
4. Success criteria: exact match, key-element coverage, human scoring, or business metrics such as ticket resolution time.

The clearer the boundaries, the more actionable the results. If your task description is “make the model smarter,” no conclusion will be usable.

Four Core Evaluation Dimensions

1. Long-Context Handling

Long context refers to how much text a model can accept in a single input. Window size is only a capacity metric—what actually determines effectiveness is whether the model can reliably locate and use key information within long text.

Common problems include:

• Recall drops when key information sits in the middle of the document—the well-known “lost in the middle” effect.
• The longer the text, the more easily the model confuses details between similar passages—for example, swapping penalty clauses between two contracts.
• When output length is constrained, the model may omit content from the latter half.

A practical test is to build “needle” cases: embed a verifiable fact at different positions in a long text, then ask about it. Record position, text length, and accuracy, and plot the curve to find the threshold where performance degrades noticeably.

2. Hallucination Rate and Verifiability

Hallucination refers to content the model generates that contradicts facts or cannot be supported by the given material. It is most common in open-ended QA and riskiest in scenarios that require cited sources.

There are two ways to reduce the risk:

• Evaluation level: include “unanswerable” test cases—questions whose answers are not in the material—and observe whether the model admits it doesn’t know. A model with poor refusal behavior will produce large volumes of confidently wrong answers in production.
• Engineering level: require the model to output supporting snippets, then programmatically verify that those snippets actually exist in the source. Flag any answer whose evidence fails verification as suspect.

3. Multimodal Understanding

Multimodal capability is the ability to process text, images, tables, and other input types together. Evaluation should focus on three things: whether visual information is read correctly, whether the correspondence between image and text is correct, and whether the model gives conservative conclusions when image quality is poor.

In practice, prepare a set of samples containing screenshots, receipts, and charts, design both recognition and reasoning questions for each, annotate ground-truth answers manually, and compute accuracy.

4. World Knowledge and Timeliness

A model’s knowledge comes from its training data, which has a cutoff date. For events after that date, the model may be entirely unaware—or it may extrapolate from similar patterns in its training data. The latter deserves special caution: models tend to apply frequently recurring historical narratives from the training corpus to new situations, producing conclusions that sound plausible but don’t match reality.

The mitigation is to integrate retrieval in scenarios that require up-to-date information, have the model cite verifiable sources, and instruct it in the prompt to distinguish “known facts” from “inferences.”

A Note on Situational Testing

Some research attempts to have models, after being told they “may be in a simulated environment,” autonomously retrieve information and judge whether their environment is real. Such tests can reveal a model’s evidence awareness and reasoning habits, but they have clear limitations.

First, the model’s judgment depends on retrieval quality. If the retrieved content is itself filtered or biased, the conclusions will drift from the facts.

Second, the model’s reasoning can be influenced by narrative patterns in its training data. When an event appears repeatedly in the training corpus, the model may treat it as a “template” for the current event and misjudge the timeline.

Third, these tests require recording the full retrieval trajectory and evidence. If only the final conclusion is saved, there is no way to tell whether the model inferred from evidence or generated the answer out of linguistic habit.

Therefore, such tests are better suited to analyzing a model’s reasoning structure than to serving as a verdict on whether “the model is lucid” or “the world is real.”

Building a Reproducible Evaluation Pipeline

The core of evaluation is fixing the test cases, model outputs, and scoring rules so that every run can be compared. Below is a simplified evaluation framework that checks key-element coverage and separately tracks “unanswerable” questions.

from dataclasses import dataclass, field
@dataclass
class EvalCase:
    prompt: str
    must_include: list[str] = field(default_factory=list)  # keywords the correct answer should contain
    unanswerable: bool = False  # True when the answer does not exist in the material
def score_case(case: EvalCase, answer: str) -> dict:
    """Score a single output, returning coverage and refusal behavior."""
    hit = [kw for kw in case.must_include if kw in answer]
    coverage = len(hit) / len(case.must_include) if case.must_include else 0.0
    abstained = any(phrase in answer for phrase in ["cannot be confirmed", "not mentioned in the material", "no relevant information"])
    return {
        "coverage": coverage,
        "abstained": abstained,
        "correct_abstain": case.unanswerable and abstained,
        "false_claim": case.unanswerable and not abstained,
    }
def summarize(results: list[dict]) -> dict:
    n = len(results)
    unans = [r for r in results if r.get("unanswerable")]
    return {
        "avg_coverage": sum(r["coverage"] for r in results) / n,
        "false_claim_rate": sum(r["false_claim"] for r in results) / max(len(unans), 1),
    }
if __name__ == "__main__":
    cases = [
        EvalCase("According to the material, what is the contract's penalty rate?", must_include=["0.1%", "daily"]),
        EvalCase("Who is the project lead mentioned in the material?", unanswerable=True),
    ]
    # In real use, the answer is generated by the model under test; fixed text is used here to demonstrate the scoring logic
    sample_answers = ["The penalty is 0.1% daily.", "The project lead is not mentioned in the material."]
    results = []
    for case, ans in zip(cases, sample_answers):
        r = score_case(case, ans)
        r["unanswerable"] = case.unanswerable
        results.append(r)
    print(summarize(results))

A few things are worth noting about this example. Keyword matching is only a rough metric—for formal evaluations, combine it with human review or semantic similarity scoring. Refusal phrases need to be extended to match your model’s actual output style, or cases will be missed. Refresh the test set regularly so the model doesn’t inflate its scores by having seen the same questions during training.

Interpreting Results and Choosing a Model

Once you have evaluation data, review it in this order:

1. Start with the error rate on unanswerable questions. If this metric is high, the model is unsuitable for factually demanding scenarios regardless of other scores.
2. Look at the long-text positional curve. If performance drops sharply when key information moves to the middle or end, consider chunked retrieval or summarization preprocessing.
3. Then check task-level metrics: extraction accuracy, human satisfaction scores, or processing time.
4. Finally, compare cost and latency. When performance is comparable, prefer the option with faster responses, lower per-call cost, and a deployment approach that fits your data compliance requirements.

Model selection shouldn’t hinge on a single ranking. The same model can behave very differently across versions and parameter settings, so evaluation should be an ongoing process, not a one-time conclusion.

Common Pitfalls

• Bigger window is always better. The window only determines how much you can input; effectiveness depends on how well the model uses that information.
• Fluent answers mean accurate answers. Language quality and factual correctness are independent dimensions.
• One case proves the point. Small samples are easily skewed by chance; cover at least hundreds of diverse cases.
• Model output can go straight to production. You need verification, human review, and log tracing to form a closed loop.

Getting Started in Practice

For development teams, a reasonable sequence is: map out business tasks and build a small test set, then set up automated scoring, then run a gray-scale validation on a subset of real data, and finally decide whether to expand usage based on the metrics. At every step, keep the inputs, outputs, scores, and version information logged for later troubleshooting.

The value of an LLM lies in whether it reliably solves a clearly defined business problem. Breaking “good to use” down into measurable indicators is what turns technology selection from impression-based judgment into evidence-based judgment.