LLM Code Generation: Post-Training, Reinforcement Learning, and Evaluation in Practice
Code generation is one of the most common and demanding applications of large language models. This article explains why code capability has become a decisive factor in whether an LLM can enter production, and how it is shaped after pre-training. It breaks down the division of labor between pre-training, supervised fine-tuning (SFT), and reinforcement learning (RL), with a focus on verifiable rewards—using real compilation and test results as objective feedback. It then lays out a rigorous evaluation framework covering functional correctness (pass@k), stability, and real-world engineering metrics, and explains how to build a sustainable data–training–evaluation loop. Finally, it offers practical engineering recommendations such as sandbox isolation, fixed and representative evaluation sets, and reporting distributions rather than single-point scores. The goal is to show how strong, reproducible evaluation and verification mechanisms deliver more long-term value than chasing a single benchmark score.
Introduction: Why “code capability” has become the dividing line for LLM adoption
Code generation is one of the most intensive application scenarios for large language models today. Whether it’s intelligent coding assistants, automated test-case generation, or internal developer productivity tools, the strength of a model’s code capability almost directly determines whether it can genuinely enter production workflows. Unlike chit-chat or copywriting scenarios, code tasks have a natural advantage: correctness can be verified automatically by a machine—the code either passes unit tests or it throws an error. This property makes code capability the best litmus test for whether post-training methodology is mature.
An emerging consensus in the industry is that code capability is not something a model “comes with” from the pre-training stage. It depends heavily on the fine-tuning of the post-training phase, especially on the quality of the reinforcement learning pipeline. Whether a model is “stable and reliable” essentially reflects whether it has built a sustainable, iterative flywheel across data construction, reward signals, training strategies, and the evaluation loop. This article systematically breaks down that topic, from principles to engineering practice.
1. Where code capability comes from: the division of labor across pre-training, SFT, and RL
To understand post-training, first clarify the responsibilities of each of the three stages:
- Pre-training: Teaches the model the general statistical regularities of language and basic programming intuition from massive corpora. It determines “how much of the code world the model has seen,” but it does not guarantee reliable output.
- Supervised Fine-Tuning (SFT): Uses high-quality “instruction–quality answer” pairs to teach the model to follow a specific task format. Its ceiling is limited by the quality and coverage of the labeled data, and it is essentially imitation learning.
- Reinforcement Learning (RL): Lets the model self-optimize in a “generate–verify–feedback” loop. It no longer just imitates a reference answer; instead, it adjusts its strategy based on how good or bad the outcome is, thereby breaking through the ceiling of SFT data.
The key reason code tasks are well suited to reinforcement learning is that the reward signal can be automated and made objective. This contrasts sharply with many open-ended generation tasks, and it is precisely the technical prerequisite for getting a “code RL pipeline” to actually run.
2. Verifiable rewards: the core engine of code reinforcement learning
The quality of reinforcement learning depends on the quality of the reward signal. In code scenarios, the industry widely adopts the approach of Reinforcement Learning with Verifiable Rewards (RLVR). Its core logic is intuitive: have the model generate code, and use the actual results of real compilation and test execution as feedback.
A minimal example of a reward function looks like this:
def compute_reward(generated_code: str, test_cases: list) -> float:"""Compute a reward signal based on unit test pass rate.Note: All candidate code must be executed in an isolated sandboxto prevent infinite loops and resource exhaustion."""passed = 0for inp, expected in test_cases:try:# Run inside a controlled sandbox with a timeout and memory limitactual = run_in_sandbox(generated_code, inp, timeout=2.0)if actual == expected:passed += 1except (RuntimeError, TimeoutError):# A failed run counts as a failure to avoid contaminating the rewardcontinuereturn passed / len(test_cases)
The value of this mechanism lies in the fact that the reward does not rely on the subjective judgment of a human-scoring model; it comes from objective execution results. This naturally resists some of the worst forms of “reward hacking,” and at the same time it makes the training signal highly scalable—as long as there are enough test cases, training samples can be generated in batches.
Of course, verifiable rewards have their limitations. When test coverage is incomplete, the model may “happen to pass” without being truly correct; poorly designed test cases can also introduce noise. So in engineering practice, it usually needs to be combined with diverse test generation and static checks to provide additional constraints.
3. How to evaluate code capability scientifically: from “looks right” to “reproducibly stable”
Evaluation is the most easily overlooked part of the post-training loop, yet it best reflects the maturity of the methodology. A rigorous code capability evaluation system usually covers several dimensions.
1. Functional correctness: the pass@k metric
The industry commonly uses pass@k to measure generation quality: sample k candidate answers for the same problem, and it counts as a pass if at least one of them passes all tests. It reflects the model’s capability ceiling better than the “hit rate” of a single generation. A sample implementation is as follows:
def estimate_pass_at_k(num_samples: int, num_correct: int, k: int) -> float:"""Estimate pass@k from combinatorial probability; num_samples is thetotal number of samples, and num_correct is the number that pass."""if num_samples - num_correct < k:return 1.0product = 1.0for i in range(num_samples - num_correct + 1, num_samples + 1):product *= 1.0 - k / ireturn 1.0 - product
In the meantime, you should report both pass@1 (correct on the first attempt) and pass@k (a hit across multiple samples): the former reflects the user experience, while the latter reflects the capability ceiling.
2. Stability: an underappreciated key dimension
Many people only pay attention to “can it get it right,” while ignoring “does it get it right consistently.” Stability has at least three layers of meaning:
- Sampling stability: For the same problem generated multiple times, is the variance of the pass rate small enough?
- Context robustness: Is it still reliable when the requirement description changes slightly or the boundary conditions shift?
- Long-task consistency: In complex engineering tasks spanning multiple files and multiple turns, does it drift into “getting more off-track the more it writes”?
A model with a high average score but enormous variance may be far less usable in production than one with a slightly lower average but extremely high stability. When evaluating, be sure to report the distribution, not just the mean.
3. Real-world engineering metrics
Beyond algorithmic benchmarks, you should also introduce metrics that closely resemble real development, such as:
- Whether the generated code passes the project’s static checks and type checks;
- Whether it introduces nonexistent dependencies or interfaces;
- Whether a patch can be cleanly applied to a real codebase;
- The compile pass rate and first-run success rate of the generated code.
4. Building a complete post-training loop: sustainable iteration is the real moat
What truly sets things apart is not the result of any single training run, but whether you can form a positive flywheel of data → training → evaluation → more data:
- Data flywheel: Mine difficult cases from real usage logs and failure samples, and construct high-quality SFT data and test cases;
- Training strategy: First use SFT to establish the basic format and capabilities, then drive reinforcement learning with verifiable rewards to break through the ceiling;
- Evaluation loop: Continuously monitor with a fixed, representative evaluation set to avoid “training once and regressing halfway”;
- Regression protection: Run the full evaluation on every iteration to promptly catch capability regressions and guarantee steady long-term improvement.
The significance of this loop is that it turns “improving code capability” from a one-time sprint into an engineered, reproducible, and measurable continuous process. Whoever masters this methodology holds the ability to keep climbing steadily along the technical path.
5. Engineering recommendations
- Sandbox first: All evaluation and training that involves code execution must run in an isolated environment with strict limits on timeout, memory, and network access to guard against security risks;
- Keep the evaluation set fixed and representative: Avoid data leakage by strictly separating the evaluation set from the training set;
- Report distributions, not a single point: Provide the mean, variance, and a breakdown across each capability dimension;
- Pair it with observability: You can combine common cloud capabilities such as logging services and monitoring alerts to continuously track key metrics along the training and inference pipeline, and to pinpoint anomalies in time.
Conclusion
Evaluating a model’s code capability should not stop at the surface level of “does a single generation look right.” Instead, it must go deeper into the maturity of its post-training methodology: whether it has built a reinforcement learning pipeline centered on verifiable rewards, whether it has a scientific and stable evaluation system, and whether it has formed a sustainable iteration loop. When these three parts are genuinely connected, the continuous improvement of code capability changes from a matter of chance into a matter of certainty. For teams looking to build or evaluate intelligent coding capabilities, getting the evaluation standards and verification mechanisms solid first is far more valuable in the long run than chasing a single benchmark score.
