LLM Post-Training: A Practical Guide for Domain Deployment and Evaluation

    Post-training is the engineering stage that turns a pre-trained base model into a domain-reliable assistant without re-running pretraining. This guide explains how to combine supervised fine-tuning (SFT), preference alignment (DPO/RLHF), and verifiable-reward reinforcement learning (RLVR) with domain RL, and how to apply them in high-stakes fields such as finance, code generation, and 3D modeling. It covers verifier design and its limits, reward-hacking and over-optimization risks, how to reduce (but never fully eliminate) hallucination, tool-calling orchestration, and a four-category evaluation framework with concrete launch thresholds and an engineering checklist.

    Post-training is the engineering stage that comes after a base model has finished pretraining. In short: post-training means applying SFT + preference alignment + verifiable-reward reinforcement learning on top of a base model, with the goal of behavior alignment and domain reliability, and without re-pretraining. It is often mentioned alongside RLVR (Reinforcement Learning with Verifiable Rewards) and domain reinforcement learning, and it is well suited to professional scenarios that demand reliable output, traceable processes, and tool calling, such as finance, programming, and 3D modeling.

    1. How to Do Post-Training: Choosing Between SFT, DPO, and Domain RL

    Post-training is not a single technique but a set of connected methods. A common engineering path is: first use SFT to teach the model basic instruction following and output formatting, then use preference alignment to improve helpfulness and safety, and finally apply domain-oriented reinforcement learning to raise performance on professional tasks.

    Method Data Requirements Reward Source Use Cases Main Risks
    SFT High-quality instruction–answer pairs Human annotation or rule-based filtering Output formatting, basic Q&A, initial behavior alignment Overfitting to annotation style, weak generalization
    DPO/RLHF Preference pairs or ranking data Preference model, rules, or human evaluation Aligning helpfulness, safety, and expression quality Reward hacking, preference noise, high annotation cost
    Domain RL / RLVR Verifiable answers, testers, or environment feedback Verifiers, unit tests, computational checks, retrieval consistency Financial computation, code generation, 3D modeling, tool calling Over-optimization, reward exploitation, task-coverage bias

    On selection: if a task has an objective answer or an automatically checkable result, prioritize designing a verifier. If a task is open-ended analysis, do not force a reliance on a single automated reward; instead, introduce rubric scoring, human spot checks, or multi-dimensional evaluation.

    2. How Post-Training Improves Domain Performance (e.g., Finance)

    In high-value scenarios such as financial analysis, financial modeling, and investment research summaries, professional tasks usually present three core challenges.

    First, cross-file / cross-document verification. Professional analysis often requires reading multiple sources at once, and the model must be able to locate sources, identify conflicts, and form an evidence chain of “citation → cross-check → conclusion.” Post-training can construct multi-document reasoning tasks so the model practices extracting evidence from source text before drawing conclusions.

    Second, structured modeling capability. Tasks such as financial metrics, valuation models, and spreadsheet calculations require the model to strictly follow templates, formulas, and formatting constraints. Post-training can use structured output samples, template constraints, and computational verification to make the model reliably produce checkable results.

    Third, a traceable research process. Professional scenarios require not only the “right answer” but also a clear explanation of “why.” Through post-training, the model can output intermediate steps, supporting sources, and recheck information, making conclusions easier to review and trace.

    On data compliance, financial scenarios must additionally emphasize: training and evaluation data must be authorized, desensitized, and compliance-reviewed, and must avoid using customer privacy, non-public information, or sensitive transaction data directly for training.

    Verifiers: Useful, but With Limits

    Verifiers are well suited to objectively decidable subtasks, such as:

    • Whether a numerical result matches a reference;
    • Whether code passes unit tests;
    • Whether a citation can be located in the source text;
    • Whether 3D modeling parameters satisfy geometric constraints;
    • Whether structured fields pass schema validation.

    But for open-ended analysis, such as “risk assessment of an industry” or “evaluation of a company’s operating quality,” it is hard to decide the answer with an automated verifier alone. Such tasks should be handled with rubric scoring, human spot checks, or a “structured intermediate steps” approach: first decompose the subjective task into verifiable steps, such as evidence extraction, metric computation, and citation verification, then subject the final conclusion to human or rule-based review.

    Also note the risks of reward hacking and over-optimization. The more reward dimensions there are, the more likely the model is to hunt for loopholes in the reward function rather than improve real task quality. Therefore, reward design should retain human auditing, adversarial samples, and a regression test set, and should continuously monitor whether metrics are “overfitting the reward.” The methods provided in this article are an engineering overview without accompanying experimental data; actual results must be validated in a specific business context.

    Hallucination Can Be Reduced, Not Eliminated

    By reinforcing verifiable correctness signals, post-training has the opportunity to reduce hallucination. But this must be validated with a consistent standard: compare the base model and the post-trained model on the same evaluation set and observe how metrics change. Common metrics include citation verifiability rate, factual error rate, and computation recheck pass rate, with specific definitions in Section 4.

    At the same time, it must be clear: hallucination can only be reduced, not eliminated. High-risk outputs in finance, healthcare, and legal domains still require human review as a safety net, especially in critical steps such as amount calculations, compliance judgments, and customer suitability.

    Below is illustrative code for post-training reinforcement learning. This code only demonstrates engineering logic; a real implementation must also handle token-level masking, length normalization, numerical stability, and distributed training.

    1. # Illustrative logic: mixed-batch sampling across tasks + in-group baseline/advantage estimation + PPO/GRPO-style update
    2. # In real engineering, batching is mixed across tasks, not serial per task with a fresh prompt fetched at every step
    3. batch_prompts, batch_tasks = sample_mixed_batch(task_pool, batch_size) # mixed batches: finance, coding, modeling, etc.
    4. rollouts = []
    5. for prompt, task in zip(batch_prompts, batch_tasks):
    6. # Sample multiple responses per prompt for the in-group baseline (similar to GRPO)
    7. group = policy_model.generate(prompt, num_samples=G)
    8. # Old-policy and reference-model logprobs at sampling time; cache and detach first
    9. old_logprobs = policy_model.logprob(prompt, group).detach()
    10. ref_logprobs = ref_model.logprob(prompt, group).detach()
    11. # Use the verifier only for objectively decidable signals; subjective tasks need rubrics or human spot checks
    12. rewards = torch.tensor([verifier.check(resp, task.reference) for resp in group])
    13. # Apply in-group baseline/normalization before computing advantage, to avoid passing raw reward straight into PPO
    14. baseline = rewards.mean()
    15. scale = rewards.std() + 1e-6
    16. advantages = (rewards - baseline) / scale
    17. rollouts.append((prompt, group, old_logprobs, ref_logprobs, advantages))
    18. # Update phase: pass in new/old policy logprobs, advantage, and clip parameters, plus a reference-model KL penalty
    19. for prompt, group, old_logprobs, ref_logprobs, advantages in rollouts:
    20. new_logprobs = policy_model.logprob(prompt, group)
    21. ratio = torch.exp(new_logprobs - old_logprobs)
    22. clip_eps = 0.2
    23. # PPO/GRPO-style clipped objective; advantages must align with the response/sample dimension
    24. policy_loss = -torch.min(
    25. ratio * advantages,
    26. torch.clamp(ratio, 1.0 - clip_eps, 1.0 + clip_eps) * advantages
    27. )
    28. # Simplified illustration: real implementation needs per-token masking, length normalization, and KL weight control
    29. kl_penalty = new_logprobs - ref_logprobs
    30. beta = 0.02
    31. loss = (policy_loss + beta * kl_penalty).mean()
    32. policy_model.update(loss)

    3. How to Land 3D Modeling and Tool Calling

    Post-training can also improve a model’s spatial understanding, multimodal understanding, and tool-calling ability. Traditional text-only models struggle to reliably handle three-dimensional spatial relationships, whereas post-training can enable a model to combine text descriptions, reference images, and parameter constraints to judge the proportions, positions, orientations, and assembly relationships between objects.

    A further step is enabling the model not just to “understand” but to “operate.” For example, if a user asks “make a desktop organizer with a drawer,” the model must first parse the intent, then call a professional modeling software interface to complete parametric modeling.

    A tool interface can be designed like this:

    1. {
    2. "tool": "create_box",
    3. "input": {
    4. "name": "drawer_box",
    5. "size": [120, 80, 40],
    6. "position": [0, 0, 0],
    7. "rotation": [0, 0, 0]
    8. },
    9. "output": {
    10. "object_id": "obj_123",
    11. "status": "success",
    12. "warnings": []
    13. }
    14. }

    A recommended engineering orchestration flow:

    1. Intent parsing: identify whether the user wants to create, modify, or query a model.
    2. Parameter confirmation: when dimensions, material, or position are missing, ask the user to supply them or provide defaults.
    3. Modeling call: select the correct tool and pass in structured parameters.
    4. Validation: check geometric legality, dimension boundaries, and component relationships.
    5. Delivery: output the model file, a preview image, or an editable object.

    Exception paths must also be handled: if the wrong tool is selected, trigger tool re-ranking or a second confirmation; if parameters are out of bounds, trigger a repair prompt or fall back to a default template; after repeated failures, hand over to a human. When commercial software is involved, attention is also needed for licenses, asset copyright, model-file storage, and compliance of secondary distribution.

    4. How to Evaluate Post-Training: 4 Metric Categories + Launch Thresholds

    Evaluating post-training should not rely on a single leaderboard score. It is recommended to establish four categories of metrics and to specify test-set sizes and launch thresholds.

    Metric Category Key Question Example Metrics Recommended Test Set Size
    General capability regression Does post-training harm general capability General Q&A, reasoning, writing, code pass rate 200–500 items
    Domain task accuracy Can it complete tasks in target scenarios Financial computation accuracy, code pass rate, modeling parameter accuracy 500–2000 items
    Reliability Is it trustworthy and traceable Citation verifiability rate, factual error rate, computation recheck pass rate 300–1000 items
    Cost efficiency Does it have launch economics Cost per thousand tasks, P95 latency, throughput Load test set + real sampling

    For the reliability metrics, it is recommended to define them clearly:

    • Citation verifiability rate: the proportion of citations that can be located in the source text and semantically match it.
    • Factual error rate: the proportion of incorrect assertions among sampled factual claims.
    • Computation recheck pass rate: the proportion of computation results that pass rule-based or human review.

    Launch thresholds should be set per scenario. High-risk scenarios should require the human-review ratio, citation traceability rate, and error rate to all meet business thresholds; low-risk scenarios can start with a canary release and then gradually expand traffic.

    On cost, distillation, quantization, and inference optimization are deployment-side optimizations that happen after training is complete, and should not be conflated with post-training itself. Post-training improves domain capability, while deployment optimization reduces delivery cost; the two should be measured separately. If internal data is available, it is advisable to provide conditional cost-reduction ranges or cases; if there is no experimental data, do not fabricate specific cost-reduction figures.

    5. Engineering Practice Checklist: From Data Compliance to Cost at Launch

    The following checklist can be used directly for project review and launch checks.

    • P0: Complete base-model selection, and record the base model’s baseline scores on the general capability set.
    • P0: Complete data authorization, desensitization, and compliance review, and clarify the sources of the training and evaluation sets.
    • P0: Filter for verifiable tasks, prioritizing tasks with objective answers that can be automatically checked.
    • P1: Design the reward function or verifier, and build a separate test set for the verifier.
    • P1: Use mixed-batch sampling across domains to avoid single-task overfitting and general-capability degradation.
    • P1: Establish an evaluation loop, incorporating the evaluation set, regression set, and failure cases into continuous monitoring.
    • P1: Run data-mix and ablation experiments to confirm how the proportions of domain data, general data, and safety data affect results.
    • P2: On the deployment side, perform distillation, quantization, and inference acceleration, and measure cost-benefit separately.
    • P2: Prepare human handover, rollback, and canary-release mechanisms to avoid full-scale launch in high-risk scenarios.

    Multi-domain joint training still matters. If the model is optimized for a single domain only, it easily overfits to that domain’s expression habits and loses generalization ability. Jointly training on multiple downstream tasks can improve domain performance while preserving general capability, but it must be paired with general-capability regression tests.

    FAQ

    Post-Training vs. Fine-Tuning vs. RAG: How to Choose?

    If fine-tuning is understood narrowly as SFT, then post-training has a broader scope, encompassing SFT, preference alignment, and domain reinforcement learning. If the task is to supplement external knowledge, real-time data, or enterprise documents, RAG is more suitable. If the task is to change model behavior, output format, tool calling, and professional reasoning flows, post-training is more suitable. In practice they can be combined: RAG provides external evidence, while post-training improves the stability with which that evidence is used.

    What Are Common Post-Training Mistakes?

    Common mistakes include: using only subjective scoring without building a verifier; optimizing a single task at the expense of general capability; a reward function that is too granular yet lacks human auditing; ignoring data compliance; lacking a consistent evaluation standard; and mistaking deployment-side cost reduction for post-training results.

    What Matters Most Before Launch?

    Before launch, focus on four things: first, whether high-risk scenarios have human review as a safety net; second, whether citations and computations are traceable; third, whether the evaluation set covers real failure modes; and fourth, whether cost and latency meet business requirements.

    Conclusion

    Post-training is becoming an important direction in the evolution of large-model capabilities. By deepening SFT, preference alignment, domain reinforcement learning, and tool calling while keeping the base model unchanged, a model has the opportunity to enter higher-value scenarios such as financial analysis, 3D modeling, and code generation. For engineering teams, the truly reusable path is not chasing isolated concepts, but building a closed loop of “verifiable tasks — interpretable rewards — multi-dimensional evaluation — human safety net — controllable cost.” You can leverage model services and compute resources offered by established cloud platforms to move toward deployment, but you must always place data compliance, reliability evaluation, and human review first.