Generative Recommendation: Distillation Ceilings and the Debate Over Reward Definition

Background: The Shift Toward Generative Recommendation

Recent advances in large language models (LLMs) have prompted a rethink of end-to-end learning across the algorithm field. Traditional recommender systems rely on a multi-stage funnel architecture: recall selects thousands of candidates from millions of items, coarse ranking narrows that set to hundreds, fine ranking applies precise scoring, and a final re-ranking stage polishes the output. This architecture is robust and engineering-friendly, but it has clear limitations: inconsistent optimization targets across stages, information loss at each step, and difficulty in jointly optimizing the entire pipeline.

Generative recommendation offers an alternative: a single unified model directly produces the final recommendation results. Given user context, the model autoregressively generates a candidate list with corresponding scores—“one model for all.” A recent technical report from a major content platform highlighted three aspects of this approach:

1. Architecture: Formulating recommendation as a sequence generation problem, where candidate generation and scoring occur within the same model.
2. Reward modeling: Defining a personalized combination score as the optimization objective.
3. Reinforcement learning: Applying an RL framework for end-to-end tuning of generated outputs.

These highlights are well-documented in the report itself. This article instead focuses on several less obvious but potentially more important implications.

A Critical Hypothesis: Generative Models as Online Distillation of Ranking Models

The P-score used in the report for both supervised signals and RL rewards is not learned directly from real user feedback. Rather, it originates from the output of the online fine-ranking model—essentially a weighted combination of indicators such as CTR and long-play rate.

This leads to a critical observation: the generative scheme is, in effect, performing online distillation of the fine-ranking model. Three points support this:

• Same supervision source: The student (generative model) learns to match the teacher’s (fine-ranking model) scoring output.
• Identical knowledge boundary: The reward depends entirely on fine-ranking output; the knowledge ceiling is fixed by the teacher.
• Explains the scaling law: Because this is a distillation task, “larger models lead to more thorough distillation” holds true, producing training curves that resemble LLM-style scaling benefits.

However, distillation has a fundamental constraint: the student’s performance is bounded by the teacher. Once the generative model has fully absorbed the fine-ranking model’s knowledge, the marginal benefit of adding parameters decays rapidly. In other words, the scaling curve for this approach likely has an inflection point near “alignment with the fine-ranking model.” Beyond that point, the ceiling is determined not by model parameters but by the teacher’s capacity.

This raises a pointed question: Is generative recommendation a true paradigm shift, or is it a more elegant engineering re-implementation of fine-ranking? If the reward remains anchored to fine-ranking outputs, the answer leans toward the latter.

Questioning the Reward Definition: Does an Indicator Combination Equal User Value?

The report defines a personalized weighted composite score, combining multiple behavioral prediction scores. From an optimization standpoint, this still boils down to optimizing the final composite score and the AUC of individual labels (CTR, long-play rate, and other offline metrics).

A dangerous gap exists here: can the “user value” we genuinely care about be fully captured by a combination of these indicators?

• Indicators are proxies, not the objective itself. High CTR does not necessarily mean user satisfaction; high long-play rates can result from clickbait or emotionally charged content.
• Weights are set empirically. The weights among various pXTR (predicted X-things-rate) signals are largely manually configured and tuned via A/B experiments, lacking a first-principles definition of value.
• Bias accumulates. Position bias, exposure bias, and popularity bias contaminate the pXTR estimates themselves. Combining them directly writes those biases into the reward.

Possible improvements fall into three categories: (1) de-biasing P-score estimates to move them closer to true preferences; (2) introducing more personalized value definitions so reward structures differ across user groups; (3) shifting from immediate behavioral metrics toward long-term value—for example, using multi-day retention or long-term consumption depth as the ultimate objective, then working backward to derive the optimal combination of intermediate indicators.

The Core Question: How to Define Reward / User Value

Viewed over a longer horizon, each architectural evolution in recommendation systems is essentially a refinement of the reward definition. This can be decomposed into two layers:

• Upstream (pretraining): Focuses on accurately estimating foundational behaviors—completion rate, like rate, long-play rate—solving the “estimate accurately” problem.
• Downstream (combination): Focuses on how to combine these base signals into an overall objective, ultimately optimizing user time spent or long-term value (e.g., 7-day lifetime value, LT7)—solving the “combine correctly” problem.

Progress on “estimate accurately” has been significant in recent years: multi-task learning, long-sequence modeling, and large-scale pretraining have steadily improved pXTR precision. But “combine correctly” remains stuck at manual weighting plus online experiments. Yet it is precisely the scientific rigor of this combination that determines the system’s ceiling and the real user experience gains—a 10% error in the combination amplifies into much larger experience loss across all upstream modules.

A simplified pseudocode illustration of the current typical approach:

# Illustration: current typical reward combination (pseudocode)
reward = (
    w_ctr   * p_ctr      # click-through rate score
  + w_eff   * p_eff      # valid view score
  + w_like * p_like      # interaction score
  + w_long * p_long      # long-play score
)
# The weights w are determined by manual experience + online A/B tuning
# Implicit problem: is "real user value" exactly a linear combination of these terms?

Comparison with the LLM Success Path: Why RLHF Doesn’t Translate Directly

Looking at LLMs clarifies the gap. The key breakthroughs came from instruction tuning and RLHF. RLHF works because of two often-overlooked preconditions:

1. The reward model is comparable in scale to the main model—it is itself a large model capable of fine-grained quality judgments on arbitrary outputs.
2. The reward definition is clear—human preference data directly labels what constitutes a good answer, with clear semantics and broad coverage.

In recommendation systems, by contrast, the reward is typically the output of a lightweight head, far smaller than the main model. The reward definition depends on combinations of existing business metrics, lacking a deep, explicit characterization of user value. The consequence: no matter how large the main model becomes, it is merely fitting a “small and vague” objective more diligently.

Therefore, if generative recommendation aims to truly replicate the LLM path, the key move is not necessarily scaling up the main model further, but rather building out the reward system: constructing a reward model comparable in scale to the main model, trained on large-scale real user feedback (rather than fine-ranking outputs), so that the reward itself has scaling ability. This may be the most worthwhile direction for the next generation of recommendation systems.

Engineering Deployment: A Phased, Pragmatic Approach

For teams currently implementing or planning generative recommendation, here are several practical recommendations:

1. Phase 1: Align with the baseline. Begin by targeting distillation of the fine-ranking model. Validate the engineering efficiency gains of the generative architecture (single-pass inference replacing the multi-stage funnel) while ensuring no degradation in user experience metrics.
2. Phase 2: Expand supervision sources. Gradually introduce signals beyond fine-ranking outputs—long-term retention experimental data, satisfaction feedback, and de-biased behavioral sequences—to loosen the teacher’s knowledge boundary.
3. Phase 3: Make the reward independent. Decouple the reward model from fine-ranking outputs, making it an independently trained and iterated module. Establish an offline evaluation standard for reward quality (e.g., causal consistency with long-term metrics).

On the infrastructure side, combined large-model and reinforcement-learning training demands significant compute and engineering pipeline investment: high-throughput distributed training frameworks to support parallel iteration of the main model and reward model, low-latency online inference services for large-model scoring, and comprehensive logging and monitoring systems to ensure a closed experimental loop. Teams with limited in-house capacity can leverage public cloud container platforms and GPU compute (for example, the general-purpose compute and container services offered by Baidu AI Cloud) to reduce trial-and-error costs and accelerate iteration cycles.

Conclusion

The greatest value of this generative recommendation technical report may not be that it provides a final answer, but that it pushes the industry toward a question that must be confronted head-on: now that architectural scaling has begun, is reward scaling ready? Architecture determines the floor; reward determines the ceiling. On the path to ever-larger models, how we define user value more scientifically and construct a reward system with comparable scale will be the true decisive factor in how far generative recommendation can go.