Practical Approaches to Synthetic Data and Reasoning Trace Reuse in Post-Training for LLMs

Why Small Models Need Reasoning Traces from Larger Teachers

In the model iteration pipeline, a recurring engineering reality is that smaller models acquire broad language capabilities during pretraining, but often lag behind same-generation larger models on tasks that require long-chain reasoning, such as mathematical problem solving, multi-step logic, and code generation. The reason is not mysterious: small models have limited capacity, and the density of high-quality reasoning samples in pretraining data is insufficient for them to internalize complex problem-solving strategies.

Post-training therefore becomes a critical compensation stage. Beyond standard supervised fine-tuning (SFT) and preference alignment (RLHF/DPO), a widely adopted industry practice is to collect reasoning traces from a stronger teacher model and inject them as synthetic supervision signals into the small model’s post-training data pool. The core assumption is that the reasoning process itself is a transferable capability carrier, whose value exceeds that of a simple final answer.

It should be noted that this practice is a common technical approach in the industry, not a unique capability of any single platform. This article takes an engineering perspective to break down the full pipeline—teacher model selection, trace collection, data cleaning, mixing schedules, and evaluation loops—and provide actionable design considerations.

Teacher Model Selection: Capability Matching Over Scale Matching

The first selection principle is not “bigger is better,” but alignment between capability dimensions and target tasks. The teacher must significantly outperform the student on the target tasks; otherwise, the collected traces are merely “detailed versions of mistakes,” amplifying noise.

In practice, evaluate candidate teachers across three dimensions:

1. Task win-rate gap: On the target evaluation set, the teacher’s pass rate should be substantially higher than the student’s (e.g., a relative improvement of 30% or more). Below that, the marginal benefit of synthetic data is insufficient.
2. Trace readability: Whether the teacher’s intermediate steps are clearly structured and parseable. Some models tend to emit verbose natural language narration, which introduces many tokens unrelated to problem solving when used directly for training.
3. Format stability: Whether the teacher can consistently produce outputs that conform to agreed templates (e.g., numbered steps, convergent conclusions). Format drift significantly increases post-processing cost.

A common mistake is to directly reuse traces across generations. A more reliable approach is: a new small model can reuse the synthetic data pipeline and architectural experience accumulated by the previous teacher, but traces must be recollected for new tasks. The reusable value lies in the pipeline methodology, not the data itself—the latter depreciates quickly as teacher capabilities evolve.

Collecting and Structuring Reasoning Traces

Collection Strategy

Trace collection is not simply asking the teacher to answer questions. High-quality collection typically involves the following controls:

• Temperature and sampling count: Sample multiple traces per question (e.g., n=4–8), retain those that pass verification, and form a “multiple solutions per problem” data shape. This helps the student learn different solution paths.
• Difficulty stratification: Bucket and collect samples by problem difficulty to avoid simple problems dominating the data pool. Simple traces carry low information density and, over time, degrade the student’s performance on hard problems.
• Rejection and truncation handling: Teachers may truncate or self-correct during long reasoning chains. Such samples must be filtered, or they will teach poor convergence behavior to the student.

Structured Representation

Raw traces need to be converted into a unified training format. A common approach is to organize them into a three-part structure: “question—reasoning step sequence—final answer,” with explicit separators between steps. Example pseudocode:

def build_trace_sample(question, steps, answer, meta):
    # steps: List[str]，teacher model's step-by-step reasoning
    trace = "\n".join(f"Step {i+1}: {s}" for i, s in enumerate(steps))
    return {
        "prompt": question,
        "response": f"{trace}\nFinal Answer: {answer}",
        "meta": {
            "teacher_task_score": meta["score"],
            "difficulty_bucket": meta["bucket"],
            "num_steps": len(steps),
        }
    }

The direct benefit of structuring is that subsequent sampling can be fine-grained by step count, difficulty bucket, or teacher score—rather than mixing blindly in an undifferentiated pool.

Cleaning Synthetic Data and Setting Quality Gates

The biggest risk with synthetic data is errors being scaled up in a systematic way. A single incorrect trace entering the data pool has a greater negative impact than a real erroneous sample, because it carries the disguise of an “authoritative teacher.” Therefore, the cleaning stage must set hard thresholds:

1. Answer consistency check: Keep only traces whose final answer matches the ground truth. For open-ended questions, use rule-based checks or a lightweight discriminator score.
2. Step rationality spot-check: Perform manual sampling review on long traces, focusing on whether there are jumps to conclusions or logical breaks between steps.
3. Decontamination: Compare against evaluation sets using n-gram or semantic similarity to remove near-duplicate samples, preventing inflated benchmark scores.
4. Distribution monitoring: Track step-length distributions and difficulty distributions of synthetic data, compare them with real SFT data, and adjust collection parameters when deviations become too large.

Cleaning is not a one-time action but should be a permanent component of the data pipeline. Teacher model updates and task set expansion will change data distributions, so cleaning rules need to be recalibrated accordingly.

Mixing Ratios and Training Schedules

Synthetic data should not dominate the post-training data pool exclusively. Engineering experience suggests that a reasonable mixing strategy typically includes three categories:

• Real human-annotated SFT data: Provides a baseline for instruction following and style alignment.
• Synthetic reasoning trace data: Supplies supervision signals for long-chain reasoning.
• General capability preservation data: Prevents capability collapse in general skills after task-specific training.

There is no one-size-fits-all ratio, but two scheduling principles can guide the design:

Principle 1: Ramp up gradually. In early training, rely mostly on real data so the model establishes stable output formats and instruction responses. In mid-to-late training, gradually increase the proportion of synthetic traces to focus on reasoning. This prevents premature overfitting to fixed templates in synthetic data.

Principle 2: Difficulty-weighted sampling. Assign higher sampling weight to traces from difficult problems, but set an upper bound to prevent a single family of hard problems from dominating gradient direction.

An illustrative curriculum schedule is shown below:

Training Stage	Real Data Ratio	Synthetic Trace Ratio	General Preservation Ratio
Early	70%	20%	10%
Middle	40%	50%	10%
Late	25%	65%	10%

The numbers above are illustrative only. Actual values should be determined through small-scale ablation experiments.

Architectural Borrowing and Transfer

Beyond data, the reuse of architectural design experience matters equally. Choices regarding attention structure, positional encoding, normalization placement, and similar design decisions within the same model generation are often already validated by large-scale experiments. When a small model is catching up, it is more economical to inherit proven architectural paradigms first, then allocate limited experimental budget to data and training strategy.

One caveat: architectural borrowing must account for nonlinear effects caused by scale differences. Certain designs that work well on large models (e.g., specific sparsification strategies) may yield negative returns on small models due to insufficient capacity. Therefore, architectural transfer should follow the order of “align the backbone first, then run small-scale ablations,” rather than wholesale adoption.

Evaluation Loops: How to Determine Whether Synthetic Data Is Actually Useful

The value of synthetic data must be demonstrated through evaluation, and the evaluation design must distinguish between “memorization” and “transfer”:

1. In-distribution evaluation: Measure pass rates on the same task families used for trace collection—this reflects fitting quality.
2. Out-of-distribution evaluation: Measure pass rates on similar tasks not included in trace collection—this reflects reasoning transfer. This is the true value metric for synthetic data.
3. Format robustness evaluation: Test model stability across prompt template variations and step count constraints, to ensure the model does not merely replicate teacher templates.
4. Regression evaluation: Monitor general capability benchmarks to confirm no capability collapse.

If out-of-distribution performance shows no significant improvement while in-distribution performance rises sharply, it usually indicates overfitting to the data pool. In that case, reduce the synthetic data ratio or expand task family coverage.

Risks and Boundaries

Finally, several engineering boundaries should be made explicit:

• Synthetic data cannot replace real feedback. Reasoning traces address process supervision, but dimensions such as preference, safety, and style still require real human feedback.
• Teacher model errors are inherited. The teacher’s capability ceiling defines the quality ceiling of synthetic data. When the teacher is upgraded, the data pool should be rebuilt accordingly.
• Data compliance must be designed upfront. Trace collection involves the source and authorization of input questions. Traceability should be defined at the pipeline design stage, not patched in afterward.

Conclusion

Synthetic data and reasoning trace reuse essentially convert the thinking process of a strong model into learning materials for a weaker model. The technical value of this pipeline lies not in any single trick, but in the engineering rigor of four stages: controllable collection, verifiable cleaning, adjustable mixing, and distinguishable evaluation. For developers, the correct order is to first stabilize the pipeline and strengthen the evaluation framework, then fine-tune ratios and explore architectural borrowing—not the reverse.