Post-Training Data Sources and Distillation Strategies: From Reasoning Traces to Synthetic Data

As LLM development enters a phase where “post-training determines the performance ceiling,” the organization of training data is undergoing structural changes. While the pre-training phase competes on corpus scale and cleaning quality, what truly separates models in the post-training phase is the ability to acquire high-quality reasoning traces and the engineering capacity to produce synthetic data. Based on publicly observable technical roadmaps, this article discusses a representative pattern: smaller models reusing reasoning traces collected from larger models during post-training, while leveraging synthetic data generated by previous-generation models to inherit capabilities, and referencing industry-proven architectural designs. This pattern involves multiple engineering steps—data collection, trace filtering, synthetic data ratio control, and architectural adaptation—warranting a systematic review.

1. Why Post-Training Relies on Reasoning Traces

Traditional Supervised Fine-Tuning (SFT) relies on human-annotated Q&A pairs. Its bottlenecks are twofold: annotation costs increase non-linearly with task complexity, and human annotators struggle to consistently produce correct, long-chain, multi-step reasoning processes. For tasks like mathematical derivation, code generation, and complex instruction following, samples that only provide the final answer offer limited help to the model; the model needs to see the intermediate “thinking” process.

The reasoning trace is a training sample format designed precisely for this purpose. It includes not only the input and final output but also structured process information such as intermediate steps, branch decisions, and self-verification. When these traces come from a more capable teacher model, a smaller model can learn these process patterns to achieve reasoning capabilities that exceed its parameter scale. This is the core value of “trace distillation”—what is transferred is not the answer, but the preferred problem-solving path.

In practice, trace distillation is not simply appending a teacher model’s output directly to the training set. Raw traces usually contain redundant explorations, incorrect branches, and stylized expressions that must be cleaned, truncated, and formatted before being used as training samples. A typical data processing pipeline can be abstracted as follows:

def build_trace_dataset(raw_traces, max_len=4096):
    samples = []
    for t in raw_traces:
        # 1. Structural parsing: separate the query, reasoning steps, and final answer
        parsed = parse_trace(t)
        if not parsed.is_complete:
            continue
        # 2. Correctness verification: filter out traces with incorrect final answers
        if not verify_answer(parsed):
            continue
        # 3. Redundancy pruning: keep the main steps and remove repetitive explorations
        steps = prune_redundant_steps(parsed.steps)
        # 4. Length control and formatting
        text = format_as_chat(parsed.query, steps, parsed.answer)
        if len(text) <= max_len:
            samples.append(text)
    return deduplicate(samples)

The key to this pipeline is not the code itself, but the trade-off criteria at each step: the strictness of correctness verification determines sample purity, the intensity of redundancy pruning determines information density, and length control directly impacts the effective context utilization rate during training.

2. Synthetic Data: Capability Inheritance and Ratio Control

Beyond reasoning traces, another crucial data source is synthetic data. Synthetic data refers to samples generated by the model itself or a previous-generation model, which are filtered and then cycled back into the training set. It serves three primary purposes in post-training:

1. Supplementing long-tail task coverage. Real annotated data is sparse for niche tasks; synthetic data can expand this on demand.
2. Achieving capability inheritance. New models can learn from the high-quality outputs of previous-generation models to stably inherit validated capabilities, thereby reducing training volatility.
3. Controlling data distribution. Synthetic data can be precisely proportioned by task type, difficulty, and language ratio, making it easier to control than real data.

However, synthetic data carries explicit risks: distribution shift and self-reinforcing bias. If synthetic data is generated entirely by the current training model without filtering, the model will continuously amplify its own preferences, leading to a drop in diversity. Common industry mitigation strategies include:

• Using a more capable teacher model for generation instead of the current training model self-generating;
• Introducing rejection sampling, retaining only samples that pass verifier or rule-based checks;
• Mixing with real data proportionally to prevent the synthetic data ratio from being too high;
• Applying deduplication and diversity metrics to synthetic data to prevent templating.

An actionable blending strategy is hierarchical mixing: real annotated data serves as an “anchor” to ensure distribution stability, reasoning trace data drives capability enhancement, and general synthetic data expands coverage. The ratio of these three must be determined through small-scale ablation experiments rather than arbitrarily fixing a value.

3. Architectural Reference: Building on Validated Designs

Beyond data strategies, architectural choices also affect post-training efficiency. A pragmatic approach is to reference industry-proven architectural designs—such as mature trade-offs in attention mechanisms, normalization positions, and expert routing—rather than exploring from scratch every time. The benefits of this include:

• Reducing the risk of training instability, as the numerical behavior of mature architectures has been extensively tested;
• Making the effects of data strategies easier to attribute, preventing interference between architectural and data variables;
• Facilitating the reuse of existing inference optimization and deployment toolchains.

It must be emphasized that architectural reference does not mean simple copying. Smaller models differ significantly from large models in terms of layers, hidden dimensions, and the number of experts; hyperparameters like attention heads and batch size need to be re-tuned. Some parallel strategies designed for large models may actually reduce utilization on smaller models. Therefore, architectural reference should focus on structural paradigms, not specific configurations.

4. Key Judgments for Engineering Implementation

When implementing the data and architecture strategies above, there are a few judgment calls that are easily underestimated:

First, teacher model selection should prioritize task fit over sheer scale. A larger-parameter teacher model is not necessarily stronger across all tasks. Especially in vertical domains like code and math, a task-specialized, medium-sized teacher might produce higher-quality traces. Model selection should be based on the pass rate on the target task’s validation set.

Second, trace data requires version management. Teacher model iterations and cleaning rule adjustments will alter data distributions. Without version records, training results become irreproducible. It is recommended to log the source, generating model version, filtering rules, and statistical characteristics for each batch of data.

Third, the verifier quality of synthetic data determines the upper limit. Rejection sampling relies on verifiers to judge correctness, so the verifier’s own accuracy directly impacts data quality. For programmatically verifiable tasks (like code execution or math answer comparison), deterministic verification should be prioritized; for open-ended tasks, multi-model cross-review must be introduced.

Fourth, evaluation must distinguish between “capability enhancement” and “style imitation.” Trace distillation makes it easy for a model to learn the teacher model’s expressive style rather than its reasoning ability. The evaluation set should include independent scoring for the correctness of intermediate steps, rather than just looking at the final answer.

5. Conclusion

Data engineering in post-training is shifting from “piling up annotations” to a systematic stage of “generating data, filtering traces, and controlling ratios.” Reasoning traces provide process-level supervision signals, synthetic data provides controllable capability inheritance and coverage expansion, and mature architectural references reduce engineering risk. The core of the strategy formed by combining these three is not mysterious; the difficulty lies in the trade-off criteria and validation loop at each step. For resource-constrained teams, a more pragmatic path is to first establish a small-scale, reproducible data pipeline and evaluation system, and then gradually scale up the teacher model size and synthetic data ratio, rather than chasing absolute data volume in one go. The controllability of data quality will always determine the final outcome of post-training more than the absolute number of data scale.