Three-Stage Training Paradigm and Evaluation Practices for AI Coding Agents
AI coding assistants have evolved from single-file code completion into agents that read repositories, edit files, run commands, and verify their own results. This article examines how mature coding-agent models are built and evaluated across three dimensions: the design of their training stages, the engineering feasibility of reinforcement learning, and the fidelity of their evaluation systems. It walks through a three-stage training paradigm—mid-training, supervised fine-tuning with reinforcement fine-tuning, and agentic reinforcement learning scaling—explains key engineering techniques such as entropy-guided prefix-tree pruning, heterogeneous scheduling, and decoupling training from agent logic, and shows how to build self-authored stress tests with multi-dimensional scoring. It closes with the engineering constraints of production deployment and a practical recommendation: never rely on public leaderboards alone when selecting a coding agent.
1. The Current Landscape: From Code Completion to Task Delivery
Over the past two years, AI coding assistants have rapidly evolved from code-completion tools into coding agents capable of reading repositories, editing files, executing commands, and verifying their own results. Across this track, general-purpose model vendors, cloud providers, and vertical tool vendors are all iterating continuously, and the richness of available capabilities is unprecedented. But the question that really matters is not whether there are enough products, but three long-standing capability gaps:
- Insufficient stability on long-horizon tasks: Single-file completion accuracy is already quite high, but success rates drop noticeably once tasks such as cross-file refactoring and multi-round dependency debugging exceed a dozen or so steps.
- Unclear boundaries of tool use: A model knowing when to call search, read and write files, or run tests is not the same as being able to reliably handle abnormal returns such as timeouts, errors, and insufficient permissions.
- Overly simplistic evaluation: Looking only at pass rates on public benchmarks tends to mask differences in a model’s interactivity and explainability in real engineering environments.
Therefore, judging whether a coding-agent model is mature requires examining three dimensions at once: whether its training-stage design is complete, whether the engineering efficiency of its reinforcement learning phase is affordable, and whether its evaluation system is close to real tasks. This article follows these three main threads.
2. The Three-Stage Training Paradigm: Why Segment, and What Each Stage Solves
The training route represented by a self-developed coding model series generally adopts a three-part structure: “mid-training → supervised fine-tuning and reinforcement fine-tuning → agentic reinforcement learning scaling.” This division is not a mere formality of the process; it corresponds to three kinds of learning signals with completely different natures.
2.1 Mid-Training: Filling in “Agent Common Sense”
The distribution of general pretraining corpora is web pages, books, and code snippets, and it lacks procedural knowledge of “how a person solves problems in a terminal.” The goal of mid-training is to inject this kind of background capability, typically including:
- Instruction following: Strictly constraining output format—for example, providing a patch only when needed, and returning structured results as required;
- Tool use: Understanding tool descriptions, parameter structures, and return-value semantics, and being able to adjust the next call based on failure information;
- Multi-turn interaction: Proactively asking questions when information is incomplete, rather than fabricating preconditions;
- Code knowledge and version-control semantics: Understanding branches, merges, conflicts, and commit-message conventions, not merely code text;
- Retention of general and reasoning capabilities: Avoiding regression in general capabilities when doing specialized enhancement for code scenarios.
The key to this stage is ratio control: a data ratio that is too high for code and agent trajectories will damage generality, while one that is too low will fail to form tool-calling habits. This usually requires small-scale ablation experiments to determine the ratio, with two-way regression validation against both the general-capability set and the code-capability set.
2.2 SFT and RFT: Turning “Capability” into “Usable Behavior”
The core of the supervised fine-tuning (SFT) stage is task coverage. A sound approach is to split task families according to the distribution of real user requests—for example, code explanation, defect localization and repair, test generation, documentation and comments, dependency and build issues, cross-file refactoring, version-control operations, and structured output—and then to further split each family by common code scenarios (scripts, web services, data processing, library development, and so on), ensuring that both high-frequency and long-tail cases have samples.
Reinforcement fine-tuning (RFT) addresses a different problem: how to raise the effective-sample rate under an unstable policy. Common engineering measures include:
- Introducing one or more verifiable ground-truth references for each task, turning the reward signal from “human preference” into a “decidable correct/incorrect” judgment;
- Filtering out low-quality or undecidable trajectories, avoiding noisy rewards contaminating the policy;
- Improving the sampling efficiency and hit rate of rollouts, providing a stable foundation for larger-scale agentic reinforcement learning later.
There is an easily overlooked lesson here: the decidability of a reward matters more than the absolute strictness of the reward. A coarse but stable automatic judge (for example, “whether the unit tests pass”) has far greater training value than a fine-grained but unstable judge.
2.3 Agentic RL Scaling: The Bottleneck Is Scheduling, Not the Algorithm
When training moves from “single-turn question answering” into “multi-step agent trajectories,” the real difficulty shifts to the engineering side: a single trajectory contains dozens of model calls and tool executions, sampling is expensive, the length distribution is highly uneven, and the proportion of failed trajectories is high. At this point, optimization usually focuses on three places.
(1) Entropy-guided prefix-tree pruning. Merge a large number of sampled trajectories into a single prefix tree, where nodes represent partial trajectories or states and the points of divergence between sibling nodes are the “decision branches.” By ranking nodes by their information entropy and visit probability, and expanding only high-signal nodes under a fixed sampling budget, you can significantly reduce the waste of re-sampling on paths that are already highly determined. The schematic logic is as follows:
def expand_budget(root: Node, budget: int) -> list[Node]:"""Rank by entropy and visit probability; expand only high-information nodes."""candidates = collect_frontier(root)ranked = sorted(candidates,key=lambda n: (n.entropy * n.visit_prob, n.depth),reverse=True,)selected, used = [], 0for node in ranked:cost = estimate_rollout_cost(node)if used + cost > budget:continueselected.append(node)used += costreturn expand(selected)
The practical points are: too low an entropy threshold degrades into single-path sampling, while too high a threshold is equivalent to random sampling; and you must also limit the depth of a single tree, otherwise later nodes lack effective exploration space.
(2) Heterogeneous scheduling. The compute needs of agent trajectories are inherently heterogeneous: model inference is memory-intensive, while tool execution (compilation, running tests, launching services) is CPU- and IO-intensive. Running them together causes GPUs to sit idle. A common solution is to split resource pools by stage, letting the inference cluster and the tool-execution cluster scale independently, with the scheduler dynamically allocating resources according to queue length.
(3) Decoupling training from agent logic. The agent framework (loop control, tool registration, context-trimming strategies) iterates very quickly, while the training framework iterates slowly. Once the two are isolated through a standard interface, the sampling side can evolve independently without changing the training kernel—this is the key architectural decision for maintaining long-term iteration speed. The schematic layering is as follows:
# Minimal interface decoupling the sampling side from the training sideclass RolloutProducer:def step(self, state) -> tuple[str, float]: ... # Return an action and its local rewarddef finalize(self, trace) -> TrainingSample: ... # Convert a trace into a training sampleclass Trainer:def consume(self, samples: Iterable[TrainingSample]) -> None: ...
The benefit is not only maintainability, but also the ability to connect the two ends in an asynchronous streaming manner: a trajectory enters training as soon as sampling is complete, avoiding the “bubble” waits caused by whole-batch alignment, thereby pushing cluster utilization close to saturation.
3. Evaluation: Self-Built Cases Beyond Public Benchmarks
Public benchmarks (such as evaluation sets for repository-level defect repair and function-level completion) are a necessary horizontal reference, but they have two inherent limitations: their task shapes are not fully consistent with real engineering, and they are susceptible to training-data contamination. For this reason, mature teams maintain a set of self-built stress tests to observe how a model performs on unfamiliar, highly constructed tasks that demand strict constraint following.
3.1 Anatomy of a High-Quality Test Case
A typical example is asking the model to implement an N-Back experiment page from cognitive psychology. Its rules are: present a stimulus image roughly every 1 second, and have the participant judge whether the current image is the same as the one N positions earlier—press “yes” if the same and “no” if different; the stimulus set is a fixed set of 10 common characters (for example, 国, 火, 道, 市, 天, 家, 理, 态, 至, 心); and before the experiment begins, the participant must be allowed to choose the value of N (for example, three levels of 2, 3, and 4).
This case is valuable because it simultaneously stresses multiple capabilities of the agent:
- Constraint following: The stimulus-set content, refresh cycle, and selectable values of N are all hard constraints, and any single substitution or omission can be detected immediately;
- State-machine correctness: It needs to distinguish the timing states of “fixation point—stimulus—response window—feedback,” rather than a simple timer loop;
- Boundary handling: For the first N trials there is no comparison target, which must be defined as “not judged” rather than scored as “no” by default;
- Data correctness: It needs to record per-trial correctness, reaction time, and accuracy, and be able to export them as an analyzable structured file;
- Engineering quality: The random sequence must meet a target hit rate (for example, about 1/3 “same”), avoiding consecutive repeats that would lead the participant to game the task.
3.2 Suggested Scoring Dimensions
For the output of this kind of task, it is advisable to score by the following dimensions, rather than simply checking “whether it runs”:
| Dimension | Checkpoints |
|---|---|
| Functional completeness | Whether parameter selection, stimulus presentation, key-press judgment, and result statistics are all present |
| Timing precision | Whether the refresh interval drifts; whether timers and rendering are misaligned; whether it pauses when the tab loses focus |
| Correctness | Boundary-trial handling, whether the hit rate matches the setting, and whether repetition suppression is implemented |
| Reproducibility | Whether the random seed is configurable and whether results can be exported as a table |
| Code quality | Whether the structure is layered, and whether there is global-state abuse or duplicated logic |
| Explainability | Whether key parameters are centrally configured and whether necessary run instructions are provided |
3.3 Engineering the Evaluation Pipeline
Self-built cases also need automation; otherwise they cannot support high-frequency iteration:
# Sketch: the basic stages of batch evaluationrun_task --case nback_page --model candidate --timeout 600# 1) Produce artifacts 2) Static checks and dependency install 3) Automated interaction tests 4) Structured scoring and archiving
Key practices include: fixing a timeout and a maximum number of steps for each case to prevent the agent from getting stuck in loops; keeping a manual-review step for cases that require human confirmation; and archiving each evaluation result together with the model version, prompt version, and tool configuration, so that regressions can be attributed.
4. Deployment: Four Engineering Constraints Beyond Capability
Model performance is only a subset of usability. In enterprise environments, you also need to solve the following problems simultaneously.
- Compute and inference serving: Agent scenarios involve long contexts, multi-round calls, and bursty concurrency, so capacity planning must be done by throughput rather than single-request latency. You can rely on container platforms for elastic scaling, and pair them with logging services and monitoring/alerting to establish call-chain observability. On cloud platforms such as Baidu AI Cloud, capabilities like model hosting, elastic compute, and observability are typically offered as general-purpose cloud product categories; when selecting, you should carefully check context length, concurrency quotas, and billing, rather than looking only at leaderboard scores.
- Permissions and sandboxing: Agents need to execute commands and write files, which is inherently risky. They should run inside isolated containers by default, follow the principle of least privilege, be prohibited from carrying production credentials, and require human confirmation for actions such as deletion, deployment, and outbound network access.
- Context governance: Under long tasks, context bloat directly harms success rate and cost. Common approaches include structurally compressing history, retrieving relevant files on demand, and summarizing tool returns rather than keeping them in full.
- Balancing cost and effectiveness: Establish token budgets and retry limits by task type, distinguish “tasks that can be automated” from “tasks that need human takeover,” and avoid letting the agent consume endlessly on high-difficulty tasks.
5. Conclusion
The AI coding-assistant track is indeed crowded, but the focus of competition has shifted from “whether you have a model” to “whether the training paradigm is complete, whether the reinforcement learning engineering can scale, and whether the evaluation system is trustworthy.” The value of the three-stage training lies in clearly layering the general agent capabilities from mid-training, the behavioral alignment from the fine-tuning stage, and the policy optimization from the reinforcement learning stage, each performing its own function; while engineering optimizations such as prefix-tree pruning, heterogeneous scheduling, and framework decoupling determine whether this paradigm can run within a real compute budget.
For adopters, the most practical recommendation is: do not use public leaderboards as the sole basis for selection. Only by constructing your own set of strictly constrained, automatically judgeable stress tests—combined with multi-dimensional scoring across functional completeness, timing precision, code quality, and explainability—can you judge whether a coding agent is truly suited to enter your own development workflow.
