From Parameter Scale to Post-Training Quality: Inside RL Post-Training for Coding and Visual Generation

1. A Paradigm Shift: Competition Moves from Parameter Scale to Post-Training Quality

Over the past two years, the logic of competition in the large model industry has undergone a structural change. Parameter scale and data quality in the pre-training stage remain the capability foundation, but once base models reach the trillion-parameter level, the marginal returns of simply “making the model bigger” diminish noticeably. What truly separates user experience is the post-training work that continues for months after release—especially continuous retraining centered on reinforcement learning (RL).

One representative industry example: after releasing a very large open-source base model, a team did not rush to start a bigger next-generation base. Instead, it did two things—externally, it kept refining developer-facing coding tooling (sandbox execution, terminal integration, browser operation, visual annotation, and more); internally, it ran joint reinforcement learning across hundreds of domains including math, code, agent tool calling, and general reasoning over roughly one hundred days. The subsequent iterative release climbed sharply in third-party blind-test arenas and comprehensive intelligence benchmarks, entered the global top tier, surpassed first-tier closed-source models on coding benchmarks, cut API pricing by about one third, and committed to releasing full weights.

Setting aside brand halo, this case reveals a repeatedly validated path:

A base model’s “full-strength skeleton” + multi-domain joint reinforcement learning + coordinated tooling refinement + a more aggressive pricing strategy = a productivity jump developers can feel

The following sections break this path into four actionable parts: the technical core, visual capabilities, evaluation methods, and cost and model selection.

2. The Technical Core: How RL Post-Training Forges “Practical Coding”

2.1 Where Base Models Fall Short

The goal of pre-training is next-token prediction. What the model learns is largely the statistical patterns of language and compressed knowledge, not the strategic ability to “finish a task in a real engineering environment.” As a result, base models are noticeably weaker than their post-trained versions in three types of scenarios:

• Long-horizon tasks: a request involving multi-file edits, dependency installation, and repeated debugging requires a closed loop of planning, execution, observing feedback, and self-correction;
• Environment interaction: executing commands in a real terminal, reading error messages, and recovering from failure states place extremely high demands on a model’s fundamentals. A common industry phenomenon is that first-generation base models may pass only around ten percent of terminal task benchmarks that emphasize command-line proficiency, leaving a generation gap versus flagship closed-source models;
• Instruction-following stability: format constraints, code style, and boundary-condition handling all require repeated calibration through post-training.

2.2 Verifiable Rewards: Making “Getting It Right” a Computable Signal

Coding tasks have a natural advantage in reinforcement learning: results can be verified automatically. Whether a piece of generated code succeeds can be judged directly by hard metrics such as unit test pass rate, compilation success, clean static checks, or terminal task completion. This is the widely adopted “verifiable reward” approach. Compared with relying on human-annotated preferences, it offers high signal density, low noise, and large-scale automated production.

There are three key engineering design points:

1. Anti-cheating rewards: models may learn to “game the system” (hard-coding test answers, skipping real logic), so randomized test cases and leakage prevention are needed in the judging environment;
2. Multi-domain joint training: training only on code tends to degrade general capabilities, so tasks such as math, code, agent tool use, and general dialogue are usually mixed to avoid narrow specialization and catastrophic forgetting;
3. Trajectory quality: the more stable the tooling (sandbox, terminal, browser), the higher the quality of execution trajectories produced by RL training, and the more reliable the strategies the model learns. This is also why “refining external tooling” and “internal RL retraining” must be driven together.

2.3 From “Can Write Code” to “Can Deliver Requirements”

After post-training, capability gains show up at three levels: higher accuracy in single-file completion; higher success rates in cross-file refactoring and defect repair; and, in agent form, end-to-end delivery “from requirement to runnable result” with sandboxes and terminals. The third level is what enterprises actually pay for, and it is the main battleground today.

3. Front-End and Visual Generation: Turning “Aesthetics” into an Optimizable Objective

3.1 Why Visual Capability Has Long Been Underestimated

Traditional code evaluation focuses almost exclusively on logical correctness, yet in real business, the “first-glance impression” of a front-end page directly affects delivery evaluation—a page that is functionally correct but has chaotic layout and unbalanced colors is simply unacceptable to the client. Recent rounds of third-party blind tests show that top models diverge more on front-end visual generation than on purely logical tasks. Aesthetics is becoming a new competitive dimension.

3.2 What Visual Generation Capability Consists Of

• Layout and responsiveness: grid systems, spacing rhythm, multi-device adaptation;
• Visual design: color systems, typographic hierarchy, whitespace and visual flow;
• Interaction and motion: hover feedback, transition animations, form validation experience;
• Complex rendering: data visualization and 3D scenes, involving open graphics standards such as WebGL, camera and lighting parameters, and shader authoring;
• Engineering quality: whether the code runs directly, whether dependencies are clear, and whether it is cross-browser compatible.

3D and visualization are the highest-barrier parts: the model must not only write runnable code but also understand spatial relationships, light and shadow, and composition—approaching “designing with code.”

3.3 How to Bring Aesthetics into the Training and Evaluation Loop

The mainstream approach: collect large amounts of human preference data, perform pairwise comparisons on two versions of a page generated for the same requirement, train a reward model, and then use reinforcement learning to distill “what humans find attractive” into the model’s policy. On the evaluation side, anonymous blind testing is emphasized—removing brand and price interference so reviewers score only the result, which yields conclusions closer to real user perception.

4. Putting It into Practice: Building Your Own Model Evaluation System

4.1 Three Limitations of Public Benchmarks

Public leaderboards have reference value, but directly guiding model selection has three problems: data contamination risk (questions may already be in the training corpus), leaderboard overfitting (vendors optimize specifically for well-known benchmarks), and distribution mismatch (benchmark questions may be far from your tech stack and requirement types).

4.2 The Four-Step Private Evaluation Set Method

1. Sampling: extract tasks from real requirement documents, historical defect tickets, and code review comments to ensure a realistic distribution;
2. Standardization: configure automatically executable acceptance criteria for each task—test cases, type checks, build scripts, page screenshot comparisons;
3. Layering: stratify by scenario and difficulty, such as completion, defect repair, cross-file refactoring, front-end page generation from scratch, and long-horizon agent tasks;
4. Blind evaluation: auto-judge logic tasks, retain human blind-review dimensions for front-end and refactoring tasks (readability, visual impression), and rotate questions periodically to prevent leakage.

4.3 Sample Code: An Automated Evaluation Loop Based on Test Pass Rate

import json, subprocess, tempfile, pathlib
TEST_CMD = ["python", "-m", "unittest", "discover", "-v"]  # judging command injected via config
def run_case(generated_code: str) -> float:
    """Write model output to a temporary workspace, run acceptance tests in an isolated sandbox, return pass rate"""
    with tempfile.TemporaryDirectory() as workdir:
        pathlib.Path(workdir, "solution.py").write_text(generated_code, encoding="utf-8")
        try:
            r = subprocess.run(TEST_CMD, cwd=workdir, capture_output=True, timeout=60)
            return 1.0 if r.returncode == 0 else 0.0
        except subprocess.TimeoutExpired:
            return 0.0
cases = json.load(open("private_cases.json", encoding="utf-8"))
scores = [run_case(generate(case["prompt"])) for case in cases]
print("Average pass rate:", sum(scores) / len(scores))

In production, sandbox isolation, timeout control, resource quotas, and test case randomization are essential.

4.4 Don’t Look Only at Pass Rate

Models with the same score can differ greatly in real experience. It is advisable to also record: average number of steps and token consumption to complete a task, end-to-end latency, self-recovery rate after failure, and the regression impact of changes on existing code. Together these metrics form a “usability profile.”

5. Cost and Selection: Making the Price-Performance Calculation Clear

5.1 Two Forms, Two Calculations

• API calls: focus on input/output token unit prices, context caching discounts, batch task discounts, and the hidden cost of repeatedly billing historical conversation in long-context scenarios;
• Self-hosted open weights: GPU memory footprint, inference throughput, operations manpower, and idle-time utilization jointly determine total cost of ownership (TCO), suitable for teams with data sensitivity, high call volume, and stable demand. Enterprises can also use model services and GPU compute provided by cloud platforms such as Baidu AI Cloud to quickly set up private evaluation and inference environments, balancing data security and elastic cost.

Recently, price competition in the industry has intensified, and “obtaining a coding experience close to flagship closed-source models at a fraction of the price” is moving from a case-by-case phenomenon to the norm—making dynamic model selection more valuable.

5.2 Hybrid Routing: Save Where You Can, Spend Where You Must

def route(task: dict) -> str:
    """Select model tier by task complexity, balancing cost and quality"""
    if task["type"] in {"completion", "rename", "comment"}:
        return "lite"        # low-cost tier: completion, rename, comments
    if task["changed_files"] > 5 or task["need_test_fix"]:
        return "flagship"    # high-capability tier: cross-file refactoring, defect repair
    return "standard"        # standard tier: single-file feature development

Paired with a cost recorder that logs each call’s tier, token consumption, and result quality (whether it passed review on the first try), you can plot your own “unit cost–quality curve” to inform tier adjustments.

5.3 Prices Change, Methods Don’t

Model pricing and capability landscapes shift every quarter. Rather than betting on whichever model is “newest and strongest,” it is better to turn the evaluation system and routing strategy into team assets: after a new release, run the private evaluation set once, update the routing table, and complete a rational selection within half a day.

6. Engineering Practice: Turning Model Capability into Real Productivity

• Context engineering: specify acceptance criteria, tech stack versions, and constraints clearly in the task description; provide a minimal reproduction and relevant files to significantly reduce wasted back-and-forth;
• Human-machine collaboration: stick to incremental diffs, review gates, and sandbox execution, and keep one-click rollback so the model operates within controlled boundaries;
• Data flywheel: feed online failure cases back as evaluation assets and distill high-quality human corrections into preference data, forming a positive loop of “use—evaluate—retrain.”

7. Conclusion

In the post-training era, the moat is no longer a single parameter scale but three things: high-quality task data, refined reward design, and an engineering closed loop between the model and its tooling. For developers, rather than chasing leaderboards, it is better to build your own private evaluation set, cost observability, and hybrid routing system as early as possible—so that when the next “high-score, low-price” model appears, you can judge within a day whether it truly fits your business.