From Parameter Scaling to RL Post-Training: Getting Flagship Coding Capability at Low Cost

1. A Paradigm Shift: From “Stacking Parameters” to “Refining Post-Training”

In the early days of the large model industry, the main competitive axis was parameter scale: from tens of billions to hundreds of billions to trillions, base models kept breaking records for the “largest open-source model.” But as the marginal returns on pre-training compute diminished, pure scale expansion became increasingly hard to deliver perceptible experience improvements. Over the past six months, the industry has seen a clear paradigm shift: rather than rushing to build a larger base model, teams are directing compute and engineering resources toward post-training, especially reinforcement learning (RL) re-training.

One representative industry case: after releasing a trillion-parameter-class open-source base, a team did not immediately follow up with a larger next-generation model. Instead, it spent roughly three months running multi-domain joint reinforcement learning on the existing base, ultimately achieving a generational leap in a third-party blind-test comprehensive intelligence index while cutting API prices substantially to a fraction of closed-source flagships. This case clearly outlines the capability formula for today’s flagship models:

Next-gen flagship model ≈ Full-power large-parameter base + Long-cycle multi-domain RL re-training + More aggressive pricing

For developers, this means a tangible benefit: flagship-level coding capability is moving from “exclusive to a few closed-source models” to a public capability obtainable at very low cost.

2. The 100-Day Iteration Cycle: Deconstructing a Typical Capability Leap

Abstracting the release process of this kind of “.1 version” reveals a highly similar path, usually divisible into four stages:

1. Base Foundation Period (T0)

Release an ultra-large-scale open-source base, validating two things: first, engineering capability—whether a trillion-parameter-class model can be trained stably and open-sourced; second, capability ceiling—whether it is competitive in “geek scenarios” such as operator optimization and compiler principles. But the practical shortcomings at this stage are usually obvious: in terminal-environment benchmarks that test real command-line fundamentals, the first-generation base often achieves only a task pass rate in the low teens, a clear generation gap versus closed-source top models.

2. Toolchain Construction Period (T0+1~2 months)

The peripheral ecosystem iterates rapidly: graphical command-line coding tools, built-in browser and visual annotation, and dual “plan/goal” interaction modes to prepare for long-cycle Agent tasks. Polishing toolchains such as sandboxes, terminals, and browsers is the prerequisite for post-training capability to be unleashed—no matter how strong the model, it needs an environment where it can “get hands-on.”

3. RL Re-training Period (T0+2~3 months)

Internally, long-cycle reinforcement learning post-training is performed on the base. This stage usually does not disclose details externally, yet it is the core of the entire leap; the next section provides the technical analysis.

4. Validation and Release Period (T0+3 months)

First, the model enters a third-party arena anonymously for blind testing, with the community completing the “stress test”; then preheating via social platform easter eggs and similar means; finally, official release with a commitment to open full weights. This rhythm of “anonymous validation → community guessing → official reveal” has become the standard playbook for open-source models to build credibility.

Pattern summary: Capability leap = infrastructure first + post-training follow-up; neither can be missing.

3. Technical Core: Why RL Post-Training Delivers Generational Gains

1. The Evolution of RL’s Role: From “Alignment Tool” to “Capability Engine”

Early reinforcement learning (represented by RLHF) mainly solved alignment problems, making model outputs conform to human preferences. But practice over the past two years has proven that RL’s value goes far beyond this—in domains where reward signals can be automated, RL can directly amplify a model’s reasoning and execution capabilities, completing the role shift from “alignment” to “capability enhancement.”

2. Verifiable Rewards: The Natural Advantage of Coding Tasks

The core reason coding and math are the domains with the greatest RL post-training returns lies in verifiable rewards:

• Whether a piece of code compiles, whether all unit tests pass, whether a terminal task achieves its goal—all can be judged automatically by programs;
• Compared with subjective human-annotated preferences, verifiable reward signals have lower noise, lower cost, and can be generated in unlimited quantities;
• The training system can let the model repeatedly “try—receive reward—correct policy” in the environment, forming an efficient exploration loop.

This also explains why “terminal-environment benchmarks” have become an important yardstick for testing model quality: what they examine is not rote answering, but the ability to interact over multiple turns in a real shell environment, invoke tools, and achieve goals—precisely the part that RL re-training is best at amplifying.

3. Engineering Challenges of Multi-Domain Joint RL

Long-cycle joint RL is not as simple as “spinning up more machines”; engineering-wise, at least three hurdles must be cleared:

• Reward design: Subjective dimensions such as aesthetics and writing require carefully constructed scoring models to avoid rewards being “gamed” (reward hacking);
• Catastrophic forgetting: Strengthening one domain may cause regression in others, requiring fine-grained data mixing ratios and regularization methods;
• Training stability: Long-cycle RL has far higher requirements for hyperparameters, throughput, and monitoring than supervised fine-tuning; any jitter in any link can waste weeks of training.

Therefore, the essence of “stacking compute instead of parameters” is to shift the center of training compute from pre-training to post-training, trading hundreds of hours of RL for a generational capability leap.

4. Evaluation Methodology: How to Scientifically Judge a Model’s Coding and Visual Capabilities

Facing an overwhelming flood of benchmark scores, developers need their own evaluation framework to avoid being misled by a single leaderboard.

1. Blind-Test Comprehensive Index: First Look at the “De-branded” Relative Position

Independent third-party evaluation organizations use anonymous comparisons and win-rate point rankings to eliminate brand halo effects. Their comprehensive indices are usually weighted across dimensions such as reasoning, code, math, writing, and instruction following, suitable as an initial screening basis—to judge roughly which tier a new model is in.

2. Terminal and Software Engineering Benchmarks: Look at “Real Work” Capability

Focus on two categories: multi-turn command-line task completion rates in real terminal environments, and repository-level bug fixing and feature implementation capability. These two benchmark types are most relevant to daily work scenarios.

3. Frontend and Visual Generation: Look at the “Aesthetic Ceiling”

Frontend code is both objective and subjective: functionality can be verified automatically, while aesthetics require human evaluation. It is advisable to establish your own scoring dimensions:

• Functional correctness: interaction logic, boundary conditions, no console errors;
• Visual expressiveness: layout, color scheme, typography, animation quality;
• Engineering quality: semantic tags, component division, style maintainability;
• Responsiveness and performance: multi-device adaptation, rendering frame rate, asset size.

5. Frontend and 3D Visual Generation Hands-On: Task Design and Acceptance Checklist

1. Why Frontend Is the “Touchstone” of Model Capability

Frontend tasks have an extremely short feedback loop—once code is generated, it can be rendered and verified, with results immediately clear. They simultaneously require engineering ability (structure and logic), design ability (aesthetics and composition), and attention to detail (animation and spacing), making them an extremely information-dense comprehensive testing ground. A common trend across recent hands-on tests is that leading open-source models can already go toe-to-toe with closed-source top models on frontend visual performance, and sometimes even lead in aesthetic preference voting.

2. Suggested Hands-On Task Set

• Single-file landing page (with parallax scrolling, dark theme);
• 3D interactive card / data dashboard (CSS 3D or WebGL);
• Reproduce a component page in a specified style from a screenshot (tests visual understanding);
• Interactive mini-game or visualization demo (tests complex logic and animation organization).

3. Prompt Template Example

Please generate a single-file index.html with the following requirements:
1. Dark theme, primary color #0F172A, accent color #38BDF8;
2. First screen is a Hero card with a 3D tilt effect following the mouse;
3. Use CSS 3D transform and requestAnimationFrame for animation;
4. Mobile responsive (breakpoint 768px), disable high-cost animations;
5. All interactive elements have hover/focus states and transition animations;
6. Semantic HTML, clear comments, no external dependencies.

4. Automated Acceptance Script Approach

def evaluate(generated_html: str, checks: list) -> float:
    score = 0.0
    for check in checks:
        # Static checks: semantic tags, external dependencies, console errors, etc.
        score += check.run(generated_html)
    return score / len(checks)

Combined with headless browser screenshots and manual aesthetic scoring, you can build a hybrid “automated functional verification + human aesthetic evaluation” pipeline at low cost.

6. The Cost Equation: How to Get the Most Value from Flagship Capability

1. Understanding the Three Levers of API Billing

• Input/output unit price: The API pricing of the new generation of open-source flagships has dropped to a fraction of closed-source flagships, so the same budget supports multiple times the call volume;
• Context caching: Repeated prefixes such as system prompts and long documents hit the cache, greatly reducing input costs;
• Batch processing: Offline tasks using batch interfaces usually receive further substantial discounts.

2. Model Routing: Save Money Without Sacrificing Quality

Tiered routing in Agent workflows is key to cost-effectiveness optimization:

def route(task):
    if task.complexity == "high":   # Architecture design, complex bug fixing
        return FLAGSHIP_MODEL       # Flagship model
    elif task.need_vision:          # Frontend visual tasks
        return VISION_STRONG_MODEL  # Model with strong vision capability
    else:                           # Completion, renaming, formatting
        return SMALL_MODEL          # Small model

Hands-on experience shows that about 70% of daily coding tasks can be handled by mid-to-low-priced models, and only 20–30% of high-difficulty tasks truly require a flagship, compressing overall costs by an order of magnitude.

3. The Hidden Value of Open Weights

Models that promise to open full weights retain a fallback for private deployment even if only the API is used in the short term, which is especially important for data-sensitive enterprises. Developers can use model service platforms provided by cloud vendors to achieve flexible access ranging from private deployment to API calls, balancing compliance, performance, and cost.

7. Implementation Recommendations: Building a Low-Cost, High-Quality AI Coding Workflow

1. Build the evaluation set before switching models: Accumulate 20–50 tasks close to your own business (including frontend visual tasks) to form an internal benchmark; run any new model through it before deciding whether to adopt it;
2. Progressive replacement: Start canary rollout from non-core links (documentation, test generation, style polishing), then gradually transition to core business code;
3. Complete Agent infrastructure: Sandbox execution, terminal and browser tools, version rollback mechanisms—the stronger the model capability, the higher the requirements for environment safety;
4. Watch the weight-opening commitment: Make “whether full weights are opened” an important evaluation item in model selection to avoid being locked in by a single vendor.

8. Conclusion

The new generation of open-source models, through roughly 100 days of RL re-training, has proven a simple truth: parameter scale determines the capability ceiling, post-training determines capability realization, and pricing strategy determines capability普及 (popularization). For developers, flagship-level coding and visual generation capability is becoming unprecedentedly cheap—the real barrier has shifted from “can you afford it” to “can you evaluate it scientifically and engineer it into production.” Rather than chasing leaderboards, it is better to start now building your own evaluation set and tiered routing workflow, and truly convert the technology dividend into productivity.