Post-Training as the Main Battleground: Capability Leaps in Finance and 3D Modeling
As foundation-model capability gains slow, the behavior layer—how a model actually performs tasks—has become the primary source of practical improvement through post-training. This article examines how supervised fine-tuning, preference optimization, and reinforcement learning advance domain-specific performance in two high-value, verifiable scenarios: finance and 3D modeling. It covers task-mix and curriculum design for multi-domain joint reinforcement learning, layered reward functions with hard constraints, guardrails against reward overfitting and catastrophic forgetting, an evaluation system built around workflow-level metrics rather than leaderboard scores, methods for measuring and governing hallucination rates, closed-loop geometric verification for 3D generation, and engineering practices for production deployment such as tool gateways, full-chain auditability, and cost-tiered routing.
1. The Base Model Stays Fixed; the Gains Are in the Behavior Layer
Across the model iterations of the past two years, one conclusion has become increasingly clear: the rate of improvement in base-model capability is slowing, while the behavior layer—the “how to do the task” dimension—still has substantial room to grow. The behavior layer includes strict adherence to format and structure, self-verification during multi-step reasoning, timing choices for tool calls, and explicit refusal or clarification under uncertainty. Most of these capabilities depend not on new pre-training corpora but on the post-training stages of supervised fine-tuning, preference optimization, and reinforcement learning.
This leads to three engineering judgments:
- A fixed base model means more controllable iteration. Behavior-layer changes can be experimented with repeatedly on the same base, with low regression-testing cost and short canary and rollback chains, making them easy to schedule and deliver by business priority.
- Multi-domain joint training beats isolated single-point fine-tuning. Some low-level capabilities are shared across domains: cross-document verification demands “locate item by item → cross-compare → provide sources,” a skill that is equally applicable to code-base retrieval, checking financial-statement footnotes, and comparing contract clauses. Joint training lets tasks reinforce one another, but it requires handling task conflicts and catastrophic forgetting.
- Value in specialized scenarios comes from traceability and verifiability, not fluency. In finance and engineering, an analysis that reads smoothly but cites no sources has business value close to zero; conversely, an answer that provides an evidence chain and whose conclusions can be verified by a program can go straight into the workflow even if plainly expressed.
2. Engineering Key Points of Multi-Domain Joint Reinforcement Learning
2.1 Task Mix and Curriculum Design
The core difficulty of multi-domain joint reinforcement learning is resource allocation. In practice, compute and sampling weights are assigned to domains across three dimensions—business value × verifiability × data availability—using a curriculum-style schedule: first establish stable reasoning and tool-calling habits on short tasks with objective answers and low judgment cost, then gradually introduce harder samples involving long documents, multiple files, and multi-step reasoning. At the same time, an adversarial set distinct from the training distribution (changing conventions, mixed units, extremely long contexts, mutually contradictory materials) must be reserved to test generalization rather than memorization.
2.2 Layered Reward Functions
A single scalar reward tends to induce shortcuts. A more robust approach is to design rewards in layers with hard-constraint priorities:
- L0 hard constraints: structural validity, field completeness, script executability. Failure is an automatic zero and does not participate in further scoring.
- L1 verifiable results: numeric consistency, formula cross-checking relationships, unit-test pass rate, geometry-check pass rate.
- L2 process compliance: whether evidence citations are complete and precisely located, whether necessary tools were called, whether key assumptions were omitted.
- L3 human preference: expert blind comparisons and pairwise scoring, used to correct expression and judgment quality that L1/L2 cannot cover.
- Penalties: unauthorized tool calls, fabricated citations, redundant verbosity, and write operations executed without confirmation.
# Illustrative: task mix and reward weights for multi-domain joint post-training# (shows the engineering structure, not a specific implementation)TASKS = {"finance": {"weight": 0.30,"reward": {"numeric_consistency": 1.0, "citation_precise": 0.6,"assumption_declared": 0.3}},"code": {"weight": 0.25,"reward": {"unit_test_pass": 1.0, "schema_ok": 0.4}},"cad_3d": {"weight": 0.20,"reward": {"geometry_check": 1.0, "script_runnable": 0.5}},"general": {"weight": 0.25,"reward": {"human_preference": 1.0}},}def score(sample, model_output):cfg = TASKS[sample.domain]if not hard_constraints_ok(model_output, sample): # L0 vetoreturn -1.0return sum(w * verifier[k](model_output, sample)for k, w in cfg["reward"].items())
2.3 Preventing Reward Overfitting and Capability Forgetting
Two common failure modes need to be guarded against in advance. The first is verifier overfitting: the policy will look for loopholes in the reward function itself (for example, piling up citation counts without checking content). The remedy is to hold out a private test set for verifiers, rotate them regularly, and reward exact “quote-to-source” matching rather than citation counts. The second is catastrophic forgetting: a small proportion of general-domain data must be continuously mixed into training (replay), with a regression check against general-capability baselines after each round, rolling back when general capability drops beyond a threshold.
3. Evaluation: From Leaderboard Scores to Workflow Metrics
Comprehensive benchmark rankings reflect general capability in an average sense, and there is often a clear gap between that and production usability for a specific business. It is advisable to establish your own evaluation system at the project-kickoff stage, covering at least four dimensions: answer correctness, evidence traceability, tool-call success rate, and per-task cost and latency. These four should each have their own threshold, so that “high overall score but unauditable” or “accurate but too slow to use” cases are not masked by an average.
More importantly, evaluation granularity must be raised from “single question-and-answer” to “end-to-end task.” For example, hand the model a several-hundred-page annual report together with a list of questions, and require it to output conclusions, a statement of conventions, citation locations (file, page, source snippet), and a list of open questions, which business experts then judge field by field. This task-level evaluation is far more sensitive to hallucination rates than question-and-answer evaluation.
4. Finance Workflow: Turning “Uncertainty” into “Verifiability”
4.1 Three High-Frequency Pain Points
- Cross-file numeric consistency: the same metric appears in the body text, footnotes, and tables under different conventions or periods, and the model easily mixes them up.
- Assumption and convention drift: base assumptions such as growth rates, tax rates, share capital, and consolidation scope are not explicitly declared, making conclusions impossible to reproduce.
- Untraceable citations: judgments cannot be traced to their sources, so manual review costs more than doing the work from scratch.
4.2 Measuring and Governing Hallucination Rates
The most quantifiable gain from post-training in specialized domains is a lower hallucination rate. The prerequisite is to define the metric clearly: use field-level judgment, meaning whether every verifiable assertion in a conclusion (number, date, entity, ratio) can find supporting evidence within the given materials; any assertion that cannot be verified is counted as a hallucination. Measure with both a fixed gold-standard set and random-sampling manual annotation, and report confidence intervals, so decisions are not based on the fluctuation of a single evaluation. Any claim that “the hallucination rate dropped by X percentage points” should be reproduced on your own gold-standard set, not taken directly from external numbers.
For governance, the most effective single step is to make output structured and require it to carry evidence:
{"conclusion": "Gross margin is expected to be under pressure, falling to 34%~36% next year","claims": [{"text": "The share of raw-material costs is rising","sources": [{"file": "annual_report.pdf", "page": 87, "quote": "Raw-material costs rose 6.2 percentage points year over year"}],"confidence": 0.81}],"assumptions": [{"name": "product_mix", "value": "maintain current-period structure"}],"open_questions": ["Next year's capacity rollout timing is not disclosed"]}
# Illustrative: assertion-level evidence verification as a pre-release gatedef verify_claims(claims, corpus):for c in claims:for s in c["sources"]:doc = corpus.load(s["file"])if not doc.contains_at(s["page"], s["quote"]):return False, f"Citation cannot be located: {s}"if c["confidence"] < 0.6:return False, "Low-confidence assertion present; route to manual review"return True, "ok"
4.3 Constraints in Financial Modeling
The key to financial modeling is not computing fast but making conventions reproducible. Practical points: consolidate assumptions into an explicit assumption dictionary (separate from the formulas); standardize units and sign conventions; apply programmatic assertions for cross-statement consistency, ratio bounds, and year-over-year/period-over-period consistency; and when the model outputs a sensitivity analysis, make clear the marginal impact of each variable on the conclusion. This way the model handles “batch construction and verification,” while humans decide “whether the assumptions are reasonable.”
4.4 Layered Routing for Human-Machine Collaboration
Handle outputs by confidence level: high-confidence conclusions that pass verification flow automatically; medium-confidence ones enter manual spot-checks after their evidence chains are presented; low-confidence content and anything involving major judgments is subject to mandatory manual review. This layering substantially lowers average review cost while keeping risk within acceptable bounds.
5. 3D Modeling: From Describing Shapes to Generating Executable Scripts
5.1 Capability Breakdown
Capabilities in 3D scenarios can be split into four layers: first, scale and proportion reasoning—estimating dimension ranges and relative relationships from text descriptions or reference images; second, spatial and assembly relationships—judging occlusion, gaps, interference, symmetry, and orientation; third, parametric and topological constraints—wall thickness, fillets, arrays, boolean order; fourth, software-dialect adaptation—translating intent into the script interfaces of a specific modeling package.
5.2 Closed-Loop Verification Is a Necessity
Generating 3D results from text most easily produces the “looks right, dimensions completely wrong” problem, so verification must be a closed loop: whether the script executes, whether the bounding-box dimensions match requirements, whether the mesh is closed (watertight), whether the face count and volume are reasonable, and whether materials and units are consistent; then a visual comparison between multi-view renders and the target reference image; and for engineering parts, a re-check of dimension annotations by reading them back.
# Illustrative: parametric generation + geometry validationSPEC = {"width_mm": 1200, "depth_mm": 600, "height_mm": 750, "wall_mm": 18}def build(spec): # Call the modeling package's script interface to generate the solid...def validate(mesh, spec, tol=1.0):sx, sy, sz = mesh.bbox_sizechecks = {"size": abs(sx - spec["width_mm"]) <= toland abs(sy - spec["depth_mm"]) <= toland abs(sz - spec["height_mm"]) <= tol,"watertight": mesh.is_closed,"wall": spec["wall_mm"] >= 3.0, # manufacturing process minimum"scale": mesh.unit == "mm",}return all(checks.values()), checks
5.3 Do Not Guess at Default Parameters
Reference images are often incomplete. The reasonable approach is to explicitly list the missing parameters with default values and ask the user to confirm before generating, rather than assuming implicitly. This is both a quality-control measure and a key to avoiding rework. At the integration level, the artifacts of a modeling task (scripts, model files, renders, validation logs) should be uniformly archived to object storage and linked to the request record, making version comparison and regression testing easier.
6. Engineering Implementation: Putting Capabilities into Production Pipelines
- Tool gateway and least privilege: all external calls go through a unified gateway with least privilege by role; irreversible operations such as writing files, submitting generation jobs, and modifying models require a second confirmation.
- Full-chain auditability: fully retain requests, model versions, prompts, tool-call parameters and returns, evidence citations, and manual-edit records, with dual SLO monitoring for quality and latency via a logging service and alerting. Heavily regulated scenarios like finance depend on this in particular.
- Cost-tiered routing: assign deterministic tasks such as extraction, classification, and format conversion to lightweight models or a rules engine, reserving high-capability models for complex reasoning; combined with result caching and batching, this typically lowers per-task cost significantly.
- Pre-release regression pipeline: a gold-standard set + adversarial set + canary comparison, with any failed gate blocking the release, to prevent the hidden regressions that post-training iteration can introduce.
- Elasticity and isolation: deploy inference services in containers, scale elastically by queue length, and isolate data and quotas between business lines to prevent cross-interference.
7. Conclusion and Recommendations
Post-training is becoming the primary source of capability gains for specialized models, and finance and 3D modeling are two typical “high-value, verifiable” scenarios within it. Three recommendations for teams:
- Bring post-training into product capability planning: define the specialized domains, acceptance metrics, and gold-standard set, rather than comparing only general leaderboard scores.
- Start with verifiable scenarios: tasks with objective answers or executable results offer clean reward signals and the fastest ROI.
- Deliver the capability trio together: model capability + tool permissions + a verification loop. Adding effort only on the model side without building the evidence chain and verification stage means the payoff in specialized scenarios will be offset by review cost.
