Post-Training and Hallucination Management for Large Models: An Engineering Evaluation Guide

    This article presents an engineering framework for evaluating post-trained large language models. It covers post-training methods and their distribution limits, MoE total versus active parameters, long-context retrieval and reasoning tests, multimodal failure boundaries, benchmark interpretation, hallucination-rate quantification, and a deployment acceptance process.

    1. Why Post-Training Is Needed After the Base Model

    Pretraining teaches a model language patterns and a large amount of knowledge from massive text corpora, but its output format, instruction following, safety boundaries, and factual reliability usually do not meet the requirements for direct production use. Post-training is the stage that addresses this gap.

    Common post-training methods include the following:

    • Supervised fine-tuning: Use high-quality instruction data written or selected by humans to teach the model how to understand task formats, such as producing structured results as required.
    • Preference optimization: Have annotators or reward models compare multiple answers to the same question and guide the model toward answers that are more helpful, accurate, and safe.
    • Reinforcement learning from verifiable rewards: In scenarios such as code passing tests, math answers that can be checked, or tool-call results that can be judged, use automated scoring signals to train the model.
    • Refusal and uncertainty calibration: Train the model to admit that it does not know when evidence is insufficient, rather than fabricating an answer.

    A key characteristic of post-training is that it adds training data and compute on top of an existing base model. As a result, capability gains tend to concentrate within the task distribution covered by the training data. Whether the model also improves on out-of-distribution tasks must be verified separately; it cannot be inferred from an overall impression.

    2. Implications and Costs of Sparse Activation Architectures

    Mixture of Experts (MoE) is a common architecture in current large-scale models. It splits the feed-forward network into multiple expert subnetworks, and a router selects a small number of experts to participate in computation for each input token.

    This architecture produces two different metrics:

    Metric Meaning Main impact
    Total parameter count Sum of all expert and shared-layer parameters Model capacity, GPU memory, and storage footprint
    Active parameters per token Parameters that actually participate in computation for each token Compute per inference, throughput, and latency

    Looking only at total parameter count or only at active parameter count leads to one-sided conclusions. Total parameter count determines how much GPU memory is needed for deployment, which often requires multi-GPU or distributed inference. Active parameter count is closer to the per-inference compute cost.

    Below is a simplified illustration of top-k routing:

    1. function route(token_hidden, gate_weights, k):
    2. scores = softmax(token_hidden * gate_weights)
    3. top_indices = indices_of_largest(scores, k)
    4. weights = normalize(scores[top_indices])
    5. return top_indices, weights

    MoE also has a common engineering issue: load imbalance. If the router always favors a small number of experts, those experts become bottlenecks while other experts remain idle for long periods. Training usually adds a load-balancing loss, and deployment also needs to monitor the hit distribution across experts.

    3. Engineering Meaning of Long-Context Capability

    The length of the context window only indicates how much input the model can accept, not whether it can use that input effectively. The two need to be evaluated separately.

    • Retrieval-style tests: Place key information at different positions in a long text and check whether the model can find it.
    • Multi-hop reasoning tests: Require the model to combine multiple pieces of information scattered across different paragraphs to complete reasoning, which is harder than simple retrieval.
    • Position sensitivity analysis: Measure the curve of accuracy as the position of key information changes. If performance is good at the beginning and end but drops明显ly in the middle, the long-context capability has a structural weakness.

    On the cost side, attention computation and KV cache memory usage both grow with input length. Practical systems usually need the following measures together:

    1. Chunk very long documents and send only the relevant chunks to the model.
    2. Summarize and compress conversation history, preserving key facts and constraints.
    3. Set resource quotas based on the distribution of business request lengths to avoid a small number of very long requests slowing down the overall service.

    4. Capability Boundaries of Native Multimodal Models

    Multimodal models generally have two implementation approaches. One shares the same representation and backbone network between text and images; the other attaches an independent vision encoder and then maps visual features into the language model space. The two approaches trade off alignment difficulty, training cost, and fine-grained recognition ability.

    Common failure modes in image understanding include:

    • Recognition errors for small text and numbers in dense tables;
    • Confusion about correspondences among multiple images;
    • Misreading chart axes, units, and legends;
    • Correct visual information but errors in subsequent reasoning steps.

    When evaluating multimodal capability, visual recognition errors and reasoning errors should be counted separately as much as possible. Otherwise, it is difficult to determine whether the vision module or the language reasoning part should be optimized.

    5. How to Interpret Benchmark Scores Correctly

    Public benchmark scores are a common reference for model selection, but their meaning is affected by multiple variables:

    • Task composition: An evaluation with the same name may contain different subsets of questions.
    • Prompt templates and few-shot examples: Format adjustments can significantly change scores.
    • Tool permissions and runtime environment: Results may differ greatly when internet access, code execution, or external tool calls are allowed.
    • Sampling parameters and number of repeats: Temperature settings and randomness across single runs introduce variance.
    • Data contamination: If evaluation questions have already entered the training corpus, scores will be higher than true capability.
    • Differences in self-reported settings: Results published by providers usually use their own conditions and may not be comparable with other models.

    When the gap between two models is only a few tenths of a percentage point, the difference is likely within the statistical error range and cannot be used to directly judge which is better. A more reliable approach is:

    1. Fix the evaluation script and prompt template, and use the same settings for all candidate models.
    2. Sample each question multiple times and report the mean and standard deviation or confidence interval.
    3. Keep raw outputs for manual review of whether scoring is reasonable.
    4. Build a separate internal evaluation set that matches the real business distribution, and cross-validate it against public benchmark results.

    6. Defining and Quantifying Hallucination Rate

    Hallucination refers to content generated by a model that is inconsistent with verifiable facts or provided materials, yet is stated with confidence. To quantify the hallucination rate, the following questions must first be clarified:

    • Should refusals count as hallucinations? Usually they should be counted separately and not included in the hallucination numerator, but they should be included in the denominator or reported separately as a refusal rate.
    • Should counting be by response or by sentence or claim? Fine-grained counting is more sensitive but has higher annotation cost.
    • What is the basis for judgment? Open-domain questions may require an external knowledge base, while questions with provided documents should use the documents as the basis.

    There are three common judgment methods:

    • Reference-answer matching: Suitable for questions with unique answers that can be automatically compared.
    • Evidence entailment judgment: Check whether each claim in the answer is supported by the given evidence, suitable for retrieval-augmented scenarios.
    • Manual sampling: Used to calibrate automatic judges and cover complex cases that automatic methods struggle to handle.

    Below is a simplified illustration of an evaluation workflow:

    1. function evaluate_hallucination(samples, judge):
    2. scored = 0
    3. hallucinated = 0
    4. refused = 0
    5. for each sample in samples:
    6. answer = generate(sample.question, sample.context)
    7. if is_refusal(answer):
    8. refused = refused + 1
    9. continue
    10. scored = scored + 1
    11. if not judge.is_supported(answer, sample.evidence):
    12. hallucinated = hallucinated + 1
    13. rate = hallucinated / scored if scored > 0 else 0
    14. return rate, refused, scored

    When reporting the hallucination rate, the sample size and number of refusals must be provided at the same time. If the original base is high, a drop to a lower level may look significant, but when the sample size is insufficient, the confidence interval may be wide and the conclusion unstable.

    Common methods for reducing hallucinations include retrieval-augmented generation, so that answers are based on retrieved materials; requiring the model to attach a citation source to each fact and validating it in post-processing; training refusal behavior; and setting confidence thresholds so that low-confidence answers are routed to human review.

    7. Selection and Acceptance Process for Engineering Deployment

    To apply the above analysis in real projects, the following steps can be used:

    1. Clarify the task and the cost of failure: Distinguish tolerable errors from unacceptable errors. For example, internal writing assistance and financial data extraction have very different accuracy requirements.
    2. Build an internal evaluation set: Cover the real request distribution, including long-tail scenarios and boundary inputs, and update it regularly.
    3. Unify evaluation conditions: Fix prompts, tool permissions, sampling parameters, and scoring rules, and compare candidate models under the same conditions.
    4. Estimate total cost: The cost per request is approximately input tokens multiplied by input unit price, plus output tokens multiplied by output unit price, plus the overhead of retrieval, retries, and human review.
    5. Canary release: First run with a small amount of traffic and compare actual quality, latency, and cost.
    6. Continuous online monitoring: Periodically sample and check the hallucination rate, track the refusal rate, timeout rate, and user feedback, and roll back or adjust promptly when metric drift is found.

    8. Conclusion

    The key to judging whether a large model is suitable for a business use case is to turn claimed capabilities into reproducible measurements. Specifically:

    • Understand which distributions post-training capability gains are concentrated in, and verify out-of-distribution scenarios separately.
    • Distinguish total parameter count from active parameter count, and evaluate deployment cost and inference cost separately.
    • Test long-context retrieval capability and reasoning capability separately.
    • Read benchmark scores under unified conditions, and pay attention to error ranges and data contamination.
    • Define hallucination clearly, and report the hallucination rate, refusal rate, and sample size together.
    • Use internal evaluation sets and canary release data as the final basis for decisions.

    As long as the evaluation process is standardized and metric definitions are clear, teams can make reliable model selection and launch decisions without relying on a single score or personal impression.