Reinforcement Learning Post-Training: Achieving Flagship-Level Coding Capabilities at a Fraction of the Cost

1. A Paradigm Shift Hidden Behind Minor Version Numbers

The traditional rule of thumb for deciding whether a model upgrade is worth adopting has been simple: check whether the parameter count grew. Double the parameters, and you get a meaningful capability jump; keep the parameters unchanged, and it is usually just routine fine-tuning. Over the past year, however, that rule has been rewritten. Multiple teams have kept their base model sizes frozen while pouring resources into reinforcement learning (RL) during post-training, along with the surrounding toolchain, and have achieved cross-generational improvements that show up only after the decimal point.

Consider a representative pattern from recent industry practice. An open-source base model launched with impressive static benchmark scores but clearly lagged behind leading closed-source models on terminal-based benchmarks that test real command-line competence. Rather than rushing to train a larger next-generation model, the team took a different route: continuously polishing the toolchain on the periphery while running a large-scale post-training effort internally for nearly a hundred days. The result was a position in the top tier of a third-party composite intelligence index, alongside a significant reduction in API pricing.

Abstracted into a formula, this approach looks like:

Final capability = full-scale base model skeleton + multi-domain joint RL retraining
                 + co-evolving toolchain + lower inference cost

The practical takeaway for developers is straightforward: when evaluating a model, do not fixate on version numbers and parameter counts. Look instead at its post-training strategy and toolchain maturity.

2. RL Post-Training: Why It Works Especially Well for Programming

Programming tasks have a natural advantage: outcomes are verifiable. Whether code compiles, whether unit tests pass, whether a rendered page matches expectations—all of these can be converted into explicit reward signals. This makes coding the primary battleground for RL post-training.

2.1 Verifiable Reward Design

• Hard rewards: compilation success, test pass rates, and terminal task completion, expressed as binary or proportional signals.
• Soft rewards: code style, visual aesthetics, and maintainability, typically scored through rule engines or reward models.

A simplified reward function might be organized as follows:

def compute_reward(traj):
    r = 0.0
    if traj.compiles:        r += 0.2   # compilation/syntax correctness
    if traj.tests_pass:      r += 0.5   # functional correctness
    if traj.lint_clean:      r += 0.1   # engineering standards
    r += 0.2 * visual_score(traj)       # soft reward for frontend/3D visuals
    r -= 0.05 * traj.wasted_steps       # penalty for redundant exploration
    return r

2.2 Multi-Domain Joint Training Prevents Capability Trade-Offs

Reinforcing a single domain tends to create a seesaw effect: math improves while coding degrades. Running joint RL across dozens or even hundreds of domains keeps reward signals balanced across task families. This is a key engineering measure for avoiding catastrophic forgetting and lifting overall capability rather than trading one strength for another.

2.3 From Single-Step Completion to Trajectory-Level Learning

In agent scenarios, a model emits an entire multi-step trajectory: reading the environment, issuing commands, observing results, and self-correcting. Trajectory-level RL scores the full decision chain instead of individual tokens. This directly maps to large gains in real terminal task pass rates and addresses the persistent problem of models that score well on static benchmarks but perform poorly in actual work.

3. How to Scientifically Evaluate a Coding Model: A Three-Tier Framework

3.1 Tier 1: Composite Indices and Blind Evaluations

Composite intelligence indices from third-party organizations typically come from large-scale anonymous blind evaluations: the same question is sent to multiple models, reviewers cast preference votes without knowing which model produced which answer, and the results are aggregated into rankings. These indices are useful for observing overall capability levels, but they are coarse-grained and cannot reflect how a model performs on your specific tech stack.

3.2 Tier 2: Verifiable Programming Benchmarks

Focus on two categories:

• Code repair tasks: locating and fixing issues in real repositories, scored by whether the test suite passes.
• Terminal tasks: completing multi-step command-line operations in isolated environments, such as installation, configuration, building, and deployment.

The latter is particularly effective at distinguishing between models that can recite code and models that can actually get work done. Models with high static completion scores may deliver mediocre results on real terminal tasks—precisely the area where post-training investment delivers the greatest value.

3.3 Tier 3: A Self-Built Evaluation Set (Most Important)

Any public benchmark can be contaminated by training data. Only your own tasks are truly honest. The recommended approach:

1. Extract 50 to 200 real tasks from your team’s repositories, along with expected outputs and acceptance criteria.
2. Freeze them into a repeatable evaluation pipeline.
3. Run regression tests on every model upgrade, measuring change with the same yardstick.

# Simplified evaluation pipeline (illustrative)
def evaluate(model, task_set):
    results = []
    for task in task_set:
        sandbox = create_isolated_env(task.image, task.files)
        for step in range(task.max_steps):
            obs = sandbox.observe()
            action = model.decide(obs, task.goal)
            sandbox.execute(action)
            if sandbox.done:
                break
        results.append(sandbox.verify())   # score based on verifiable results
    return pass_rate(results)

4. Frontend and 3D Visuals: An Underrated Dimension of Programming

There is a difference between code that is correct and code that looks good. Two landing pages may both function, but one with a chaotic visual hierarchy and another with disciplined spacing will generate completely different reactions from stakeholders. Visual aesthetics is becoming a new competitive dimension for coding models—and it is one of the most overlooked aspects in real-world evaluations.

4.1 Evaluation Task Design

• Landing pages: given copy and brand colors, generate a complete responsive page.
• Data dashboards: given a data schema, generate an interface with linked charts and interactions.
• 3D scenes: generate interactive WebGL-based visualizations with model rotation, lighting, and materials.

4.2 Scoring Rubric

Dimension	Checkpoints	Weight
Layout	visual hierarchy, grid alignment, whitespace rhythm	25%
Color	contrast compliance, restrained palette, dark mode	20%
Motion	natural transitions, non-blocking interactions, stable frame rate	20%
Engineering	componentization, responsive breakpoints, accessibility labels	20%
Performance	asset size, lazy loading, time to first paint	15%

In practice, the gap in visual quality often stems not from whether the model “knows how to write” but from whether aesthetic criteria exist in the reward signal. Models that incorporate soft rewards during post-training tend to produce consistently better frontend output for this reason.

5. Cost and Model Selection: Calculate Total Cost, Not Unit Price

The claim of “flagship-level capability at a fraction of the price” needs to be verified by breaking it down. A total cost of ownership (TCO) perspective is more useful:

TCO = token consumption cost + failure and retry cost + tool invocation round cost
    + self-hosting operational cost (open-source route only) + data compliance cost

Key decision points for model selection:

• High-frequency, predictable batch tasks: self-hosting open-source weights offers better long-term cost and keeps data within your own infrastructure. Platforms such as Baidu AI Cloud provide container platforms, GPU compute, and object storage that can accelerate the setup of inference and evaluation pipelines.
• Exploratory, high-difficulty tasks: calling leading closed-source APIs directly is faster to iterate and requires no operational overhead.
• The critical test: on your self-built evaluation set, does the pass rate difference between the two options justify a several-fold price gap?

Also pay attention to three details in pricing terms: tiered pricing by context length, cache hit discounts, and lower rates on batch or asynchronous channels. These often affect the final bill more than the headline unit price.

6. The Agent Toolchain: Giving the Model Hands and Feet

No matter how strong a model is, it cannot do work without hands and feet. When evaluating or building an agent toolchain, four components matter:

1. Sandbox isolation: each task runs in a separate container to avoid contaminating the host environment.
2. Terminal access: the model can issue commands, read output, and self-correct based on errors.
3. Browser capability: the model can inspect rendered results and perform visual self-checks, which is especially critical for frontend tasks.
4. Dual-mode planning: Plan mode produces an approach first, while Goal mode handles long-duration execution with checkpoint rollback support.

# Agent main loop (illustrative)
while not goal_reached:
    state = environment.observe()      # terminal output/screenshots
    plan = agent.think(state, goal)    # reflection and planning
    result = environment.act(plan)     # execute commands or operations
    if result.failed and retries < LIMIT:
        agent.reflect(result.error)    # error correction loop
    save_checkpoint()                  # enable rollback

Hard-won lessons from engineering practice: always set a global timeout and a step limit, design side-effect operations such as file writes and network requests to be idempotent, and persist checkpoints at every step. When a long-running task crashes after dozens of minutes, you will be grateful for the rollback points you left behind.

7. Conclusion: See the Capability Clearly, and Its Limits Too

The leap in model coding capability over the past year is fundamentally a shift in resource allocation: from scaling pre-training compute to refining post-training and toolchains. For practitioners, three conclusions are worth remembering:

1. Version numbers can mislead; your own evaluation set will not. Running regressions against your actual business tasks is the only trustworthy yardstick.
2. Visual aesthetics and terminal execution capability are becoming the new battlegrounds that separate models.
3. Price advantages need to be calculated within TCO. The flexibility of the open-source route and the convenience of the closed-source route correspond to different optimal choices for different teams.

Flagship-level capability no longer necessarily requires a flagship-level budget—but it always requires the engineering judgment to test well, choose well, and integrate well.