Post-Training for Large Models in Practice: A Guide to Evaluating Coding and 3D Vision Skills

From Parameter Scaling to Post-Training Refinement: The New Competitive Landscape

Over the past two years, the dominant narrative in large model development has been parameter count—the larger the base model, the stronger its core capabilities. However, a clear inflection point has emerged. Leading teams are shifting release cadence from annual major versions to quarterly minor releases. These so-called “x.1” versions often deliver near-generational improvements without changing parameter size.

Consider a recent example from the open-source community: a team released a large-scale open-source base model, then instead of immediately training a larger successor, spent roughly one hundred days applying reinforcement learning (RL) post-training. The resulting version climbed to the top tier of global comprehensive intelligence benchmarks in third-party blind tests, approaching leading closed-source models, while simultaneously cutting API prices by about one-third.

This case offers two valuable takeaways for developers:

1. The gap between open-source and closed-source is narrowing from a “base model gap” to a “post-training gap.” The ceiling of a base model is determined by compute and data, but post-training offers an extremely high return on investment—the shortest path for the open-source community to catch up.
2. “Small versions, big leaps” will become the norm. When evaluating whether to migrate to a new model, version numbers matter less than the depth of post-training and verifiable evaluation evidence.

Let’s break this down from a technical perspective: what exactly happens during post-training, and how can coding and visual capabilities be scientifically evaluated?

Unpacking RL Post-Training: What Happens in a Hundred Days

Three Stages: From Imitation Learning to Autonomous Exploration

Modern large model capability development typically follows three steps:

• Pre-training: Next-token prediction on massive corpora determines the model’s “knowledge breadth” and “linguistic intuition.”
• Supervised Fine-Tuning (SFT): Using high-quality instruction-response pairs to teach the model to “follow instructions”—essentially imitation learning.
• Reinforcement Learning (RL): The model repeatedly attempts tasks in an environment, optimizing its policy based on reward signals, which determines the “upper bound of its performance.”

The ceiling of SFT is data quality—models can only imitate good answers they’ve seen. RL, however, allows the model to explore solutions that surpass the demonstrations. This is why recent capability leaps generally come from the RL stage rather than larger SFT datasets.

Verifiable Rewards: Why Coding Is One of the Best-Fit Domains for RL

The core challenge of RL is sourcing rewards. Open-ended tasks like dialogue or writing rely on reward model scoring, which can be gamed through scoring loopholes. Coding tasks, by contrast, naturally provide a clean class of reward signals—verifiable rewards:

• Whether all unit tests pass;
• Whether the code compiles and builds successfully;
• The exit code and output of a script in a real terminal;
• Whether a patch is accepted by a repository’s continuous integration pipeline.

These signals are objective, reproducible, and hard to falsify, enabling RL to scale broadly in the coding domain. One intuitive piece of evidence: many base models score only in the low teens on terminal command-line benchmarks—because such tasks require the model to execute multi-step operations, read environmental feedback, and recover from errors, precisely the weaknesses of models without RL training. Versions re-trained with months of RL often achieve several-fold improvements on these benchmarks.

Multi-Domain Joint Training: Preventing “Lopsided” Skills and Forgetting

Another engineering consideration is the breadth of training tasks. If RL is applied only to coding, the model risks regression in other capabilities (catastrophic forgetting) or developing a “skewed” coding profile—excelling at algorithmic puzzles while struggling with real-world engineering. Best practices involve building a joint reward system covering over a hundred domains:

• Code (algorithms, frontend, data pipelines, terminal operations);
• Reasoning and mathematics;
• Tool invocation and long-horizon agent tasks;
• Instruction following, safety, and output format stability.

Joint training also yields a hidden benefit: capability transfer. RL on tool use and long-horizon tasks improves the model’s performance in real IDEs and terminals, since these are fundamentally sequence-decision problems requiring multi-step execution, feedback handling, and self-correction.

Implications for Developers

Rather than asking “how large is the parameter count,” ask three questions: Which task domains does the post-training cover? Are the reward signals verifiable? Is there evaluation evidence from terminal-level and repository-level real tasks?

Evaluating Coding Ability: From “Checking Scores” to “Building a Pipeline”

Three Benchmark Types, Three Capability Layers

Level	Task Format	Capability Assessed	Limitations
Function-level generation	Complete functions based on comments/signatures	Language mastery, algorithmic fundamentals	Far from real-world engineering; easily saturates
Repository-level repair	Locate and fix bugs in a real repository	Code retrieval, context management, multi-file editing	Evaluation sets are prone to training data contamination
Terminal/Agent tasks	Complete multi-step development or operations in a sandbox terminal	Planning, environment interaction, error recovery	Higher setup cost, but best discriminative power

Recommendation: prioritize the third layer, supplemented by the second. Terminal-style tasks align most closely with daily development work—running commands, installing dependencies, debugging environments—and best predict real-world experience.

Four Principles for Building Custom Evaluation Sets

Public benchmarks get “gamed”; a team’s best defense is a self-built evaluation set:

1. Tasks sourced from real tickets: Sample from de-identified bug reports and feature requests from the past six months, rather than hand-written toy problems.
2. Automated judgment criteria: Each task must include executable judging commands (tests, type checks, static analysis), with outcomes decided by machine-verifiable results.
3. Multiple sampling: Each task should be sampled at least 3–5 times, reporting pass rates and variance—a single pass may just be luck.
4. Freeze and rotate: Refresh the evaluation set every six months to prevent newer models from having “seen” your tasks during training.

A Minimal Working Evaluation Framework

The following skeleton, roughly fifty lines, can be directly integrated into a team’s existing test infrastructure:

import subprocess, tempfile, shutil
from dataclasses import dataclass
@dataclass
class Case:
    task_id: str
    prompt: str      # Task description including repository context
    judge_cmd: str   # Command to run unit tests and return exit code
    timeout: int = 600
def run_once(client, case: Case) -> dict:
    ws = tempfile.mkdtemp()
    try:
        resp = client.generate(case.prompt, temperature=0.2)
        write_files(ws, resp.files)  # Write model output to sandbox
        proc = subprocess.run(case.judge_cmd, shell=True, cwd=ws,
                              capture_output=True, timeout=case.timeout)
        return {"id": case.task_id, "pass": proc.returncode == 0,
                "tokens": resp.usage.total_tokens}
    finally:
        shutil.rmtree(ws, ignore_errors=True)
def evaluate(client, cases, n=5):
    rows = [run_once(client, c) for c in cases for _ in range(n)]
    by_id = {}
    for r in rows:
        by_id.setdefault(r["id"], []).append(r["pass"])
    return {
        "pass_any":  sum(any(v) for v in by_id.values()) / len(by_id),
        "pass_all":  sum(all(v) for v in by_id.values()) / len(by_id),
        "avg_tokens": sum(r["tokens"] for r in rows) / len(rows),
    }

The two metrics each carry distinct meaning: pass_any reflects “can solve with multiple attempts” (suitable for scenarios with automatic retries), while pass_all reflects “gets it right the first time” consistency (suited for unattended pipelines). avg_tokens serves as input for the cost analysis in the next section.

Frontend and 3D Vision: Building a Scorecard for “Aesthetics”

Why Visual Taste Is a Hard Requirement

Users cannot perceive whether backend code is well-written, but they can immediately perceive frontend quality. For “production-oriented programming” scenarios, a model’s visual taste is not a bonus—it’s a hard requirement. Recent rounds of third-party blind tests also show that the gap between newer models is shifting from “can they write it” to “does it look and feel good, and is it usable?”—aesthetics and interaction completeness have become the new dividing line.

Five Quantifiable Evaluation Dimensions

While aesthetics appear subjective, they can be decomposed into judgeable dimensions, scored on a 0–2 scale by reviewers:

1. Layout and hierarchy: Is the grid aligned? Is whitespace reasonable? Is the information hierarchy clear?
2. Interaction completeness: Are hover, focus, loading, empty, and error states all handled?
3. Responsiveness: Does the layout break across three common viewport sizes?
4. Performance and accessibility: First-screen rendering, animation frame rate, semantic HTML tags, and keyboard accessibility.
5. Code maintainability: Component decomposition, style organization, and avoidance of unnecessary dependencies.

Recommendation: two reviewers score independently and average the results, with screenshots for reference to reduce subjective drift.

3D Capabilities: A High-Discrimination Litmus Test

3D scene generation simultaneously tests spatial reasoning, mathematics (camera, projection, lighting), and rendering API proficiency—making it one of the most discriminative task types in recent years. The acceptance checklist should include: drag-to-rotate, scroll-to-zoom, double-click-to-reset; at least two switchable lighting presets; explicit fallback handling when the rendering environment is unavailable; and single-file delivery with no external dependencies for easy review archiving.

A ready-to-use prompt template (applicable to any model for horizontal comparison):

Generate a single-file HTML page implementing an interactive 3D product showcase:
1. Use WebGL rendering with mouse drag-to-rotate and scroll-to-zoom;
2. Provide "soft/high-contrast" lighting presets, switchable with one click;
3. If the rendering environment is unavailable, fall back to a static schematic;
4. Inline all styles and scripts without referencing any external resources;
5. Maintain smooth frame rates while using simple geometric shapes.

Feed the same template to different models, score against the rubric in section 4.2, and you’ll have a compelling side-by-side comparison within twenty minutes.

Cost-Benefit: Flagship Capabilities ≠ Flagship Bills

The Unit-Price Illusion: Real Cost Is Determined by Four Variables

Looking only at the per-million-token price can seriously mislead cost estimates. The real cost is approximated by:

Per-task cost ≈ (input tokens × input price + output tokens × output price)
               × (1 + retry rate) × (1 + rework rate)

• Retry rate: A weaker model has a lower unit price but a worse first-attempt pass rate; after two or three retries, the total cost exceeds the stronger model.
• Rework rate: Manual rework time typically costs more than ten times the token cost.
• Context length: Repository-level tasks can involve hundreds of thousands of input tokens, sharply amplifying the weight of input pricing.
• Output verbosity: Wordy models produce more explanatory text, directly inflating the bill.

This is why “near-flagship coding ability at a fraction of the flagship price” is disruptive—not just because the unit price is low, but because post-training raises the pass rate, which simultaneously lowers both the retry and rework multipliers.

Four General Cost-Reduction Measures

1. Prompt caching: In multi-turn agent sessions, system prompts and repository context are highly repetitive. Enabling caching significantly reduces input costs.
2. Task routing: Route simple completions and format conversions to smaller models; reserve architecture design and cross-repository refactoring for flagship models.
3. Batch processing: Send non-real-time tasks like offline evaluation or batch code review through batch channels, which typically offer substantial discounts.
4. Output trimming: Constrain output format in the system prompt (e.g., “output code only, do not restate the question”), directly reducing output tokens.

Scenario-Based Model Selection Matrix

Scenario	Key Requirement	Selection Recommendation
IDE inline completion	Latency-sensitive	Cost-effective small model; prioritize cache hits
Single-file feature development	First-attempt pass rate	Mid-tier model + automatic retry
Repository-level refactoring	Long context, planning capability	Flagship model; worth paying for pass rate
Visual/3D prototyping	Aesthetics and interaction completeness	Test empirically with a scorecard, don’t rely on leaderboards
Pipeline batch processing	Cost-sensitive	Low-priced model + strict judging; escalate failures to human review

A Four-Week Rollout: Building a Team-Level Evaluation and Routing System

• Week 1 — Build the evaluation set: Sample 50–100 tasks from real tickets and attach automated judging commands; meanwhile, record 10 frontend/3D vision tasks.
• Week 2 — Integrate models: Connect 3–4 candidate models in parallel through a unified model service interface (major cloud platforms, such as Baidu AI Cloud, commonly offer unified multi-model access, logging services, and monitoring alerts), and get the evaluation scripts running.
• Week 3 — Dual review: Machine metrics (pass rate, stability, token consumption) combined with human blind review (visual scorecard) to form a candidate matrix per scenario.
• Week 4 — Routing and monitoring: Deploy routing policies by scenario, configure alerts for success rate, cost, and latency, and place the evaluation set on a quarterly rotation schedule.

The key mindset shift: model selection is not a one-time procurement decision but a continuously operating evaluation pipeline. Leaderboards tell you who to test; your own evaluation set tells you who to use.

Conclusion

Returning to the opening example: roughly one hundred days of RL post-training propelled an existing base model to the global forefront without increasing parameter count—while cutting prices by a third. This reinforces a judgment that is becoming consensus: the competitive focus of large models has shifted from the “arms race” of pre-training to the “fine craftsmanship” of post-training. For developers, this means two things. First, the cost of acquiring top-tier coding capabilities is rapidly declining, and there is no reason for teams not to integrate stronger, cheaper models into their workflows. Second, distinguishing signal from noise matters more than chasing trends—building your own evaluation sets and scoring systems is the only way to stay clear-eyed through the next wave of “x.1” releases.