How Large Programming Models Are Trained: From Mid-Training to Agentic Reinforcement Learning

    This article breaks down how modern large programming models are trained, stepping back from individual product launches to examine the general methodology, evaluation systems, and real-world validation practices behind them. It walks through the three-stage training pipeline—mid-training, supervised and rejection-sampling fine-tuning (SFT & RFT), and agentic reinforcement learning scaling—explains the engineering techniques used to make large-scale agentic RL cost-efficient, and outlines how to evaluate coding models through software-engineering benchmarks and hands-on tasks. It closes with a practical framework for enterprises looking to assess and adopt AI coding assistants.

    Introduction: Why the AI Coding Assistant Space Keeps Getting More Competitive

    Over the past two years, intelligent coding assistants have evolved from simple “code completion plugins” into agents capable of understanding natural-language requirements, autonomously invoking tools, and completing complex engineering tasks across multiple turns of dialogue. Nearly every team with large-model R&D capabilities is investing in this direction: on one side, multimodal models keep iterating; on the other, models focused on code generation and software engineering are being released in rapid succession. This raises a question that comes up again and again—has the AI coding assistant space become too crowded?

    To answer it, rather than fixating on the benchmark numbers from any single release, it’s better to return to the fundamentals: how exactly is a genuinely useful programming model “trained”? Where are its capability boundaries? And how should enterprises evaluate and adopt it rationally? Instead of reviewing any specific product, this article dissects the general training methodology, evaluation systems, and real-world validation practices behind modern large programming models, helping readers build a transferable framework for technical judgment.

    1. The Complete Training Pipeline for Large Programming Models

    A mature programming model is rarely trained in a single step; it goes through staged, progressively deepening phases. The approach most widely accepted in the industry can be broken into three stages: Mid-Training → Supervised Fine-Tuning & Rejection-sampling Fine-Tuning (SFT & RFT) → Agentic RL Scaling. Each stage has a distinct role: the first lays the foundation, the second aligns to tasks, and the third pushes up the ceiling.

    1.1 Mid-Training: Building the Foundation for Agentic Capabilities

    Mid-training sits between general pre-training and alignment training. It is an often-overlooked but critical stage. Its core goal is to gradually transform the model from “a well-read language model” into “an agent candidate that understands tools, can interact, and can get things done.”

    Specifically, this stage typically strengthens the following capabilities:

    • Instruction following: enabling the model to accurately understand and execute complex, multi-constraint instructions rather than giving irrelevant answers.
    • Tool use (Function/Tool Calling): the heart of a coding agent is “knowing how to use tools”—reading files, running tests, executing commands, retrieving documentation. Mid-training must teach the model to issue the right tool calls at the right time and correctly parse the returned results.
    • Multi-turn interaction: real software development is conversational and continuous, so the model must stay aligned with the goal and remember past decisions across long contexts.
    • Code knowledge injection: supplementing domain knowledge such as language features, framework APIs, and common engineering patterns to reduce “hallucinated APIs.”
    • Git and PR-related skills: generating well-formed commit messages, understanding diffs, writing Pull Request descriptions—abilities that stay close to real development collaboration workflows.
    • General and reasoning capabilities: specialization must not come at the cost of general logical reasoning and common-sense understanding, or the model will “overfit one point and collapse across the board.”

    The value of mid-training lies in building a solid base: by the time alignment and reinforcement training begin, the model already has the foundational skills needed to perform agentic tasks, which significantly improves learning efficiency.

    1.2 Supervised Fine-Tuning & Rejection-sampling Fine-Tuning: Aligning to Real Tasks

    The goal of the second stage is to make the model “match what users actually want.” This stage is usually split into two steps.

    In the Supervised Fine-Tuning (SFT) stage, teams systematically collect high-frequency user tasks and typical code scenarios—such as code completion, bug fixing, refactoring, unit test generation, code explanation, and cross-file modification—and construct high-quality “instruction–response” sample pairs. The key here is not having “more” data but having “better” data: the quality, diversity, and difficulty distribution of the samples directly determine how well the model covers real needs.

    The Rejection-sampling Fine-Tuning (RFT) stage then optimizes further on top of SFT. Its core idea is this: for the same task, have the model generate multiple candidate trajectories (rollouts), introduce multiple reference answers (ground truth) for comparison and filtering, and keep only the truly high-quality samples for training. The benefits are twofold:

    • Higher rollout efficiency: using reference answers as a filtering criterion avoids wasting compute on ineffective samples.
    • More stable downstream reinforcement learning: a high-quality initial policy distribution lets the RL stage converge faster and more stably.

    1.3 Agentic RL Scaling: Squeezing Training Efficiency to the Limit

    Once a model has foundational agentic capabilities, the real improvement in its ceiling comes from Agentic RL. But agentic RL faces a harsh reality: a complete agent trajectory often requires “multiple rounds of reasoning + multiple tool calls + long-chain reasoning,” so its sampling cost is far higher than that of ordinary RL. Without engineering optimization, training clusters sit idle much of the time and compute is wasted.

    To address this, the industry has developed several representative engineering approaches to compress training costs at scale:

    • Entropy-guided trajectory tree pruning: first organize massive agent trajectories into a “prefix tree”—a multi-branch structure with shared prefixes—then rank nodes by entropy (a measure of a branch’s uncertainty and information content) together with visit probability, expanding only the “high-signal” branch nodes within a limited compute budget and pruning large numbers of redundant, low-value explorations.
    • Heterogeneous scheduling with decoupled training and agent logic: fully decouple “model training” from “agent execution logic,” and use a unified scheduler to flexibly allocate tasks across heterogeneous compute (for example, a mix of training and inference cards), achieving “zero-bubble” high utilization and keeping the cluster as fully utilized as possible.

    The point of this combination is to greatly increase training throughput per unit of compute while preserving the effectiveness of reinforcement learning, which makes “larger-scale, higher-quality” agentic RL feasible. This is also the key engineering support that enables step-change improvements in programming model capabilities.

    2. How to Evaluate: Look Beyond the Leaderboard

    Once a model is trained, you must answer “is it actually any good?” The industry generally uses two evaluation methods.

    2.1 Standardized Benchmarks: Software-Engineering Benchmarks

    The most important evaluation for programming models is benchmarks oriented toward real software-engineering tasks (what the industry often calls “solving real repository issues”). These benchmarks typically provide a real project’s codebase along with a problem description, and assess whether the model can locate the problem, modify multiple files, and make the tests pass. This is closer to engineering reality than traditional “function-level code generation” and is also harder.

    That said, a reminder: benchmark scores are only one reference dimension. Different models perform very differently across difficulty tiers, languages, and frameworks, and a single high score does not mean a leading overall experience.

    2.2 Real-World Validation: Testing in Real Scenarios

    More convincing than a leaderboard is running a task that has real complexity and verifiable correctness. A classic example is the N-Back experiment page from cognitive psychology, which is well suited as a “programming capability stress test” because it simultaneously exercises requirement understanding, state management, timing logic, and interaction design.

    How the N-Back experiment works: a stimulus flashes every 1 second, and the user judges whether the current stimulus is the same as the one N steps earlier—click “Yes” if the same, “No” if different. The stimuli are drawn from a set of 10 common characters, for example [“国”, “火”, “道”, “市”, “天”, “家”, “理”, “态”, “至”, “心”]; at the start of each experiment, the user can choose the value of N (for example, 2, 3, or 4).

    A ready-to-use prompt example:

    1. Build an N-Back cognitive experiment web page with the following requirements:
    2. 1. At the start, let the user choose the value of N (offer three options: 2, 3, 4);
    3. 2. Every 1 second, flash a stimulus randomly drawn from these 10 characters:
    4. ["国","火","道","市","天","家","理","态","至","心"];
    5. 3. The user judges whether the current stimulus matches the one N steps earlier:
    6. click "Yes" if it matches, "No" if it doesn't;
    7. 4. Record the user's accuracy and reaction time, and show summary statistics
    8. when the experiment ends;
    9. 5. Keep the interface clean, keep the timing accurate, and support restarting.

    When using tasks like this for validation, focus on four dimensions:

    • Requirement understanding: whether the model grasps the semantics of “N steps earlier” as a relative index, rather than simply comparing with the immediately previous stimulus;
    • State correctness: whether it correctly maintains a history-sequence buffer and initializes it properly at the start of the experiment;
    • Logical rigor: whether boundary conditions among timing, judgment, and scoring are handled properly;
    • Interaction completeness: whether it produces a complete, runnable, reusable page in one pass, rather than fragmented snippets.

    In real development, it is best to design prompts to be clear, verifiable, and driven by acceptance criteria, so you can objectively distinguish differences between models.

    3. Adopting AI Coding Assistants: A Rational Framework for Enterprises

    Back to the original question—is the space too crowded? My view is that fierce competition shows precisely that the value of this direction has been thoroughly validated; but for users, what really matters is “how to choose and how to use,” not “who released a new model.”

    For enterprises and teams, I would suggest keeping the following in mind when evaluating and adopting:

    1. Separate “capability” from “marketing”: a model’s claimed benchmark results should be validated against your own business scenarios, especially your programming languages, frameworks, and coding standards.
    2. Pay attention to engineering support: a good coding assistant depends on stable inference services, controllable context management, and integration with your R&D workflow (for example, code hosting and CI/CD).
    3. Leverage general cloud capabilities: on the deployment and operations side, you can build a sustainable coding-assistant system on top of general capabilities such as object storage (for datasets and model artifacts), container platforms (to elastically host inference services), and logging services and monitoring/alerting (to observe call quality, latency, and anomalies). Baidu AI Cloud can provide solid underlying support for deploying and operating large programming models through these kinds of general cloud product capabilities.
    4. Build an evaluation loop: turn real business tasks into an internal evaluation set and run continuous regression tests, so you are not misled by a single demo.

    Conclusion

    On the surface, competition among large programming models is about “whose score is higher,” but at its core it is a combined contest of training methodology and engineering efficiency: mid-training sets the foundation of agentic competence, supervised and rejection-sampling fine-tuning determine the quality of task alignment, and scaled agentic reinforcement learning determines the capability ceiling. Once you understand this pipeline, together with standardized benchmark evaluation and real-scenario validation, you can stay clear-headed amid the noise—unmoved by any one-off product launch, and focused only on the ability to solve real engineering problems. That may be the most rational stance toward a “crowded” space.