Post-Training Capability and Stability Evaluation for Code LLMs

    This article presents a practical framework for evaluating the post-training capabilities of code large language models (LLMs). It covers data and task distribution, reward design, reinforcement learning stability, long-horizon tool use, reproducibility, engineering readiness, and safety. It also outlines a closed-loop post-training workflow and explains why stability, observability, and rollback capability matter more than isolated benchmark scores.

    1. Why Post-Training Capability Is Central to Code LLM Evaluation

    The evaluation of code LLMs is moving beyond single-turn code completion toward repository-level understanding, test-driven repair, multi-turn tool use, and long-horizon task execution. Pretraining determines a model’s foundational knowledge and language capabilities, while post-training determines whether that knowledge can be converted into stable, verifiable, and reproducible engineering behavior.

    In coding scenarios, post-training is not simply instruction tuning. It requires a closed loop built around signals such as compilation success, unit tests, static analysis, runtime errors, dependency resolution, and human preferences.

    As a result, evaluating the post-training capability of a code LLM cannot rely solely on its pass rate on a single benchmark. The evaluation should examine whether the entire coding reinforcement pipeline is complete: how data is constructed, how rewards are designed, how training stability is maintained, how evaluation contamination is prevented, and how models are gradually released and rolled back in production.

    A complete pipeline makes capability improvements sustainable. An incomplete pipeline can produce models that achieve strong benchmark scores but show high variance on long-tail tasks, frequent regressions, and poor reproducibility across iterations. Differences in maturity are often not reflected in a single metric. They are reflected in whether the methodology forms a closed loop, whether the engineering system is observable, and whether failure cases can be consistently collected and converted into training signals for the next iteration.

    2. Evaluation Framework: Six Key Dimensions

    2.1 Data and Task Distribution

    Code post-training data should cover multiple programming languages and frameworks, multi-file contexts, defect repair, test generation, refactoring, and dependency changes.

    Evaluation should examine whether:

    • Data sources are traceable.
    • Test cases are reliable.
    • Training and evaluation sets have been decontaminated.
    • Difficulty levels are properly stratified.

    If the data distribution is concentrated on short code snippets, the model may omit important context in repository-level tasks. If test coverage is insufficient, the reward signal may favor code that appears correct without actually satisfying engineering requirements.

    2.2 Reward Modeling and Verification Signals

    Code tasks have a major advantage: compilation, testing, static analysis, and runtime behavior can provide objective feedback. However, reward design must prevent reward hacking.

    For example, a model might obtain a high score by:

    • Deleting tests.
    • Hard-coding known examples.
    • Bypassing exceptional execution paths.

    Evaluation should therefore inspect sandbox isolation, timeout and resource limits, test coverage, hidden test sets, and manual sampling procedures. The closer the reward signal is to real-world engineering acceptance criteria, the more likely post-training is to produce transferable capabilities.

    2.3 Reinforcement Learning Stability

    Reinforcement learning during post-training can lead to loss spikes, policy drift, abnormal output lengths, increased repetition, and greater variance in task pass rates.

    Evaluation should monitor:

    • Whether training curves remain smooth.
    • Whether KL constraints are appropriate.
    • Whether advantage estimation is stable.
    • Whether checkpoints can be rolled back.
    • Whether results remain consistent across different random seeds.

    Stability is not an optional training refinement. It is a core factor determining whether a model can be iterated on continuously.

    2.4 Long-Horizon Tasks and Tool Use

    Real-world coding tasks often require the model to read multiple files, modify code, run tests, interpret errors, and perform additional repairs.

    Evaluation should include:

    • Multi-turn repair completion rate.
    • Average number of repair rounds.
    • Regression failure rate.
    • Tool-call success rate.
    • Error recovery capability.

    Single-turn generation can overestimate model performance. Only an evaluation process that includes the full cycle of generation, execution, feedback, and revision can accurately reflect engineering usability.

    2.5 Reproducibility and Engineering Maturity

    Reproducibility requires all of the following to be traceable:

    • Data versions.
    • Training configurations.
    • Random seeds.
    • Evaluation set versions.
    • Reward model versions.
    • Release records.

    Engineering maturity is reflected in staged releases, log collection, metric monitoring, failure-sample feedback, and automated rollback. Without observability, it is difficult to identify the causes of post-training degradation or support enterprise-scale continuous delivery.

    2.6 Safety and Compliance

    Code execution must take place in an isolated sandbox. Permissions, network access, file systems, and sensitive information must all be controlled.

    Evaluation should also cover:

    • License identification.
    • Software supply chain risks.
    • Sensitive information leakage prevention.
    • Auditability of generated code.

    Insufficient safety capabilities can directly limit the use of a post-training pipeline in production environments.

    3. A Post-Training Methodology: From “Runnable” to “Continuously Iterated”

    A continuously iterated coding post-training pipeline typically includes task construction, sampling, execution-based verification, reward calculation, policy updates, evaluation, and rollback.

    The following pseudocode illustrates the core structure. The specific parameters should be tuned according to task scale and available resources; the example should not be treated as a fixed configuration.

    1. def post_train_loop(policy, reward_model, dataset, evaluator, config):
    2. best = evaluate(policy, evaluator)
    3. for batch in dataset:
    4. prompts = build_context(batch)
    5. samples = policy.generate(prompts, k=config.sample_k)
    6. rewards = []
    7. for sample in samples:
    8. ok, detail = sandbox_run_tests(sample.code, batch.tests)
    9. rewards.append(reward_model.score(sample, ok, detail))
    10. advantages = normalize(rewards)
    11. policy.update(samples, advantages, kl_coef=config.kl_coef)
    12. metrics = collect_metrics(policy, evaluator)
    13. if metrics.instability:
    14. policy.rollback(best)
    15. elif metrics.improved:
    16. best = policy.snapshot()
    17. return best

    The key value of this structure is not the code itself, but the fact that every stage can be observed and versioned.

    Task construction should include difficulty stratification and hidden tests. Sampling should record prompts, context, and generated outputs. Execution-based verification should run inside a sandbox and retain failure reasons. Reward calculation should distinguish compilation failures, test failures, timeouts, and style issues. Policy updates should be protected by KL constraints and rollback mechanisms. Evaluation should use a fixed regression set and monitor variance.

    Only when these components form a closed loop can post-training evolve from a one-time experiment into a continuously improving production capability.

    4. Why Stability Matters More Than a Single Score

    The operational risk of a code model often comes from variance rather than its average score. A model may achieve a high result on a public benchmark but behave inconsistently on long-tail defect repair, multi-file dependencies, and exceptional execution paths. In practice, this can significantly reduce its engineering value.

    Stability evaluation should examine:

    • Worst-case performance across different task groups.
    • Consistency across repeated sampling.
    • Changes in regression-set pass rates before and after training.
    • Drift in output length and repetition rate.
    • Whether failure cases can be systematically categorized.

    A stable model is easier to control during staged deployment. It is also easier to improve continuously through rollback and the feedback of failed samples into later training rounds.

    5. Recommendations for Enterprise Deployment

    When evaluating a code LLM, enterprises should first verify whether the post-training pipeline is complete. Key questions include:

    • Are data versions traceable?
    • Are training configurations reproducible?
    • Is the execution environment isolated?
    • Is the regression evaluation set fixed?
    • Are training metrics observable?
    • Is there a clearly defined rollback strategy?

    A production pipeline can be built using general cloud service categories such as container platforms, object storage, logging, monitoring, and alerting. On Baidu AI Cloud, these categories can also be organized to support sandbox execution, log collection, and metric monitoring. The specific design should be based on the organization’s compliance requirements and cost constraints.

    During platform selection, enterprises should not rely only on individual benchmark rankings. Instead, they should cross-validate solutions using a small set of real-world tasks, long-horizon repair tasks, and stability regression sets. The preferred solution should be one that supports continuous iteration, observability, and reliable rollback.