Post-Training with Reinforcement Learning: Approaching Flagship Coding Capability at Lower Cost

Beyond Parameters: Post-Training Is the Model’s Second Growth Curve

For a long time, the industry judged large models by parameter count. But the iteration practice of the past two years has produced a more important conclusion: the base model sets the floor, post-training sets the ceiling. Once a base model reaches the trillion-parameter level, the marginal returns from simply scaling up diminish quickly, while systematic post-training can lift overall performance by what amounts to a generational jump.

Watch for a clear industry signal: a growing number of “.1”-style version updates no longer rely on larger parameters, but instead direct resources toward two areas:

1. Refining the surrounding toolchain: sandbox execution environments, terminal access, browser tools, code retrieval, and similar components continue to mature, giving models “hands and feet” so they can turn ideas into runnable results;
2. Deepening post-training: with reinforcement learning at the core, models are systematically retrained over months around verifiable signals from real tasks.

The cost-effectiveness of this approach has been repeatedly validated: the same model skeleton, after dozens to hundreds of days of reinforcement learning retraining, can improve its aggregate performance on public benchmarks by more than ten points, sometimes accompanied by simultaneous API price reductions—capability rising while price falls, and the gap between the two curves widening. For users, this means “getting near-flagship capability at a fraction of the price” is moving from marketing language to an accountable selection logic.

Reinforcement Learning Post-Training: From “Solving Problems” to “Getting Things Done”

2.1 SFT Teaches Form, RL Trains Function

Supervised fine-tuning (SFT) is fundamentally imitation: the model is given large numbers of “question—standard answer” samples and learns the shape of responses. Its ceiling is locked by the quality of the demonstration data—behaviors absent from the demonstrations are hard for the model to invent on its own. Reinforcement learning takes another path: the model is placed in an executable environment to explore autonomously, and task outcomes serve as reward signals to update the policy. This brings two key changes:

• The model can move beyond the boundaries of demonstration data and discover solutions humans have not written down;
• Rewards are anchored directly to “whether the task is completed” rather than “whether it looks like the standard answer,” making them closer to real productivity.

2.2 Programming Is an Ideal Training Ground for RL: Rewards Are Naturally Verifiable

The biggest bottleneck in reinforcement learning is reward design, and for programming tasks the rewards are almost free and objective: whether syntax and compilation pass, unit test pass rates, the number of static check warnings, and for front-end tasks, rendering fidelity. A typical reward function skeleton looks like this:

def compute_reward(task, submission) -> float:
    # First gate: failing compilation is an immediate loss
    if not compiles(submission, lang=task.lang):
        return -1.0
    reward = 0.2
    # Functional correctness: test pass rate carries the most weight
    passed, total = run_tests(task.tests, submission)
    reward += 0.5 * (passed / max(total, 1))
    # Engineering quality: readability, duplication, warnings
    reward += 0.2 * quality_score(submission)
    # Front-end tasks: visual fidelity
    if task.kind == "frontend":
        reward += 0.1 * visual_similarity(render(submission), task.design)
    return reward

2.3 Multi-Domain Joint Curriculum: Specialized Gains Without General Regression

Pushing reinforcement learning hard in a single domain easily makes a model lopsided: code gets stronger while common sense and writing degrade. The mature approach is multi-domain joint training—mixing tasks from hundreds of domains such as coding, math, reasoning, and writing according to a curriculum ratio, dynamically balancing learning progress across domains so that specialized capability advances while the general foundation is preserved.

2.4 A Qualitative Shift in Long-Horizon Task Capability

What best demonstrates the value of reinforcement learning is multi-step, long-horizon tasks: in a real command-line environment, installing dependencies, locating faults, modifying code, running tests, and delivering results. A base model without sufficient post-training often loses track of the goal or repeats actions after a dozen or so steps; a fully retrained model can reliably maintain task context and complete the entire loop. This kind of “agentic” capability is almost exclusively obtainable through post-training.

Evaluation Methodology: A Four-Layer Funnel for Judging Real Productivity

Leaderboards are someone else’s ruler; your business is your own exam room. A four-layer funnel is recommended for evaluating a coding model.

3.1 Layer One: Aggregate Indices—Watch Trends, Not Absolute Values

Third-party aggregate intelligence indices and anonymous head-to-head human blind tests are suitable for quick cross-comparison and can expose inflated “peak-at-launch” results. But two caveats apply: benchmark data may be contaminated by training sets, and high-scoring domains may barely overlap with your business. This layer is for initial screening only.

3.2 Layer Two: Function-Level Code Benchmarks for Fast Baseline Screening

Have the model complete function-level algorithm and implementation problems under controlled conditions, and measure the pass@1 rate. It is cheap and fast, suitable as a regression test, but still far from real engineering.

3.3 Layer Three: Repository-Level Engineering and Terminal Tasks, Closer to Real Work

• Fix-type tasks: build evaluations from historical defects in real open-source repositories (“given the problem → submit a fix”), examining cross-file understanding and engineering conventions;
• Terminal-type tasks: in a sandbox, give the model a command-line environment and a goal (for example, “get a certain service running locally”), examining multi-step planning, troubleshooting, and self-correction.

These two task types mark the watershed between “benchmark scores” and “productivity.” Some models that initially struggled on terminal tasks achieved several-fold improvements after reinforcement learning retraining—a direct demonstration of post-training’s value.

3.4 Layer Four: Front-End Visuals and 3D—Structuring the Subjective Dimension

Aesthetics cannot be automatically scored, but they can be structured. Score rendered results across six dimensions: layout stability, color and contrast, typography and hierarchy, interaction feedback, motion detail, and responsive behavior; for 3D scenes, add three more: reasonableness of scene organization, performance controllability (frame rate and rendering load), and intuitiveness of camera and interaction.

def evaluate(endpoint, tasks):
    report = {"codegen": [], "terminal": [], "visual": []}
    for t in tasks:
        if t.kind == "codegen":
            out = call_model(endpoint, t.prompt)
            report["codegen"].append(pass_at_1(out, t.tests))
        elif t.kind == "terminal":
            report["terminal"].append(run_in_sandbox(endpoint, t.goal, max_steps=50))
        elif t.kind == "frontend":
            shot = render_screenshot(call_model(endpoint, t.prompt))
            report["visual"].append(rubric_score(shot, t.rubric))
    return summarize(report)

3.5 The Ultimate Standard: Build Your Own Internal Benchmark

Extract 50–200 real tasks from your own codebase (historical defects, shipped pages, common refactors) and freeze them into an internal evaluation set. Any model upgrade or architecture decision should be measured against it—this is the only ruler that marketing language cannot bend.

Front-End Visuals and 3D: The Underestimated Efficiency Lever

Why “looking good” is hard productivity: in front-end delivery, there is a huge gap between “it runs” and “it ships”—layout, color, spacing, and motion each consume rework time. If a model can generate a page close to shippable in one pass, it saves design reviews and multiple revision rounds; 3D scenes even more so, where manually tuning lighting and camera parameters can take half a day.

Common failure modes can be organized into an acceptance checklist: layout collapse (content overflows and misaligns as soon as there is more of it), random color (no color system, uncontrolled saturation), fake responsiveness (layout breaks as soon as the window shrinks), and 3D performance failure (loading everything at once, frame rate halved).

The “render—screenshot—compare” loop is the most effective evaluation and iteration method: render the generated code in a headless browser, screenshot it and compare against the target, score it manually or with a vision model against a rubric, then feed the results back to the model for self-correction. Browser-in-the-loop is becoming standard architecture for front-end agents.

The Cost Ledger: From “Unit Price” to “Total Cost of Ownership”

5.1 Three Faces of API Billing

• Base token unit price: input and output are priced separately, and output is often several times more expensive;
• Context caching: frequently used system prompts can be heavily discounted when they hit the cache;
• Batch channels: non-real-time tasks go through batch interfaces, usually with substantial discounts.

Hidden costs must also be counted: retries and failed requests, token inflation from long contexts, and manual rework caused by hallucinations. One “cheap mistake” can be more costly than ten “expensive correct” calls.

5.2 The Real Bill for Open-Source Self-Hosting

Open weights give enterprises the option of self-hosting, but “open source” does not mean “free”: GPU purchase or rental, the manpower for inference optimization (quantization, compilation, scheduling), and the operations for availability and autoscaling all cost real money. A rule of thumb: only when daily inference volume reaches a considerable scale and task patterns are stable is the total cost of self-hosting likely to fall below calling APIs directly. With managed model hosting, elastic compute, and inference acceleration capabilities offered by Baidu AI Cloud, enterprises can also choose a middle path between “fully managed APIs” and “pure self-hosting,” scaling elastically with business peaks and troughs while balancing data boundaries and cost control.

5.3 Quick Decision Reference

Scenario	Recommended Path
Early validation / interactive development	Call APIs directly, pay as you go
High-frequency batch tasks	Caching + batch channels to lower marginal cost
Large-scale stable traffic	Managed deployment or self-built inference cluster
Strong compliance / data sensitivity	Private deployment, keeping weights and data on the internal network

Turning Evaluation Results into Productivity: Integrate into Real Workflows

Three integration forms: in-IDE assistants (completion, refactoring, explanation—fastest to show results); command-line agents (autonomously completing the “set up environment—modify code—run tests” loop in a sandbox); and browser-in-the-loop front-end agents (generate → render → screenshot → self-evaluate → revise, fully automated page iteration).

Plan / Execute dual modes: for complex tasks, first have the model output a plan (task breakdown, file list, risk points), and after human confirmation, move to the execution state. The planning state can be revised repeatedly at very low cost, avoiding the waste of “running a hundred steps in the wrong direction” during execution.

while not done and steps < MAX_STEPS:
    obs = env.observe()                       # terminal output / file tree / page screenshot
    action = model.plan(goal, history, obs)   # plan the next action
    result = env.execute(action)              # execute inside the sandbox
    history.append((action, result))
    done = env.check(goal)                     # acceptance criteria are authoritative

Engineering guardrails: dual caps on steps and tokens to prevent runaway execution; tests, compilation, and rendering checks as hard acceptance gates; full trace logging and replay of action sequences; per-task token consumption tracking to translate “efficiency gains” into accountable numbers.

Understanding the Boundaries: Post-Training Is Not a Silver Bullet

Even with thorough retraining, stay clear-eyed about capability boundaries: very long-horizon, multi-service large refactors will still expose planning weaknesses, and the closer a task is to “change ten places at once,” the lower the automation success rate; for new frameworks and interfaces that appeared after the training cutoff, models tend to confidently fabricate, so the acceptance step must provide the backstop; visual aesthetics have an industry average line, but the “last ten percent” of brand guidelines still requires human review. Bind every selection conclusion to your own internal benchmark—someone else’s high score does not mean your efficiency.

Conclusion

The lesson from this round of technical evolution is clear: the base model sets the floor, post-training sets the ceiling, evaluation determines selection, and cost determines scale. As reinforcement learning retraining becomes the main theme of version upgrades, the most important thing for users is not to chase every new name, but to build their own evaluation funnel and cost ledger, so that every model upgrade can be translated into quantifiable engineering gains.