Rethinking Coarse Ranking in Search & Recommendation: Architectures and Evolution

The Role and Engineering Constraints of Coarse Ranking

In modern search and recommendation systems, the overall ranking pipeline is typically divided into cascaded stages: recall, coarse ranking, precise ranking, and re-ranking. This funnel-like architecture is designed to balance computational resource consumption against final business outcomes. The coarse ranking module sits in a critical position between full-scale recall and the high-complexity precise ranking stage, serving as a bridge that connects the two.

The primary business function of coarse ranking is to perform an initial filter on the massive candidate set produced by the recall stage—often ranging from thousands to tens of thousands of items—and compress it to a few hundred. This reduction reserves sufficient computational budget and feature space for the downstream precise ranking model.

Due to the fact that the candidate volume handled by coarse ranking is significantly larger than that of precise ranking, its engineering constraints are distinctly different:

1. Extremely high throughput requirements: Within a strict end-to-end latency budget of tens of milliseconds, coarse ranking must complete scoring computations for thousands of candidates.
2. Limited computational resources: In online deployment, the coarse ranking model cannot rely on the dense use of complex cross-attention networks, attention mechanisms, or ultra-long feature sequences as precise ranking models do.

Constrained by these factors, the evolution of industrial coarse ranking models has followed two representative paths: the early high-efficiency two-tower structure, and the more recent exploration of lightweight networks with complex feature crossing capabilities.

The Two-Tower Structure: The Industrial Cornerstone of Coarse Ranking

The two-tower architecture, exemplified by the DSSM (Deep Structured Semantic Models) model, is the most widely applied foundational structure for coarse ranking. Its core design philosophy is “decoupled computation,” where user-side features and item-side features do not interfere with each other for the vast majority of the model’s forward propagation.

1. Architecture Principles and Workflow

The workflow of a two-tower model can be divided into offline and online phases:

• Offline Phase: The model is trained using massive amounts of historical user behavior logs. The item-side tower receives item attributes, statistical features, and other data, mapping them into fixed-dimensional vectors through multiple layers of a DNN. Online, the underlying infrastructure (such as a generic vector retrieval engine) pre-computes all candidate items into embeddings and stores them in memory or high-speed cache via an index.
• Online Phase: When a user request arrives, the system extracts user features in real-time, inputs them into the user-side DNN, and computes the user embedding on the fly. This user vector is then used to calculate a dot product or cosine similarity against the candidate item vectors obtained from the recall stage, producing the coarse ranking score.

2. Engineering Advantages of the Two-Tower Structure

The separated structure offers unparalleled advantages for online serving. Because item-side vectors can be fully pre-computed offline, the online phase only requires a single user-side DNN inference. The remaining scoring process is transformed entirely into pure matrix multiplication operations. When dealing with vast candidate sets, this design can fully leverage the instruction set optimizations of low-level vector computation libraries, ensuring extremely fast response times online.

3. Bottlenecks in Expressive Power

The critical weakness of the two-tower model stems from the same “separation” design. Since user and item features only interact via a dot product at the final layer, the model cannot capture fine-grained feature crossings at lower levels.

For example, complex cross-features, such as the deep crossing between “categories of items the user has historically clicked” and “the category of the current candidate item,” cannot be effectively implemented in a two-tower model. Placing such cross-features into one side forcibly breaks the condition for item vector pre-computation. Placing them into both sides leads to a dot product operation that is insufficient to express high-order non-linear crossings. This directly results in the relatively weak expressive power of two-tower models, often leading to a “recall ceiling” problem.

4. Optimization Practices for Two-Tower Models

To mitigate the weak expressiveness issue, industrial practitioners often employ optimization techniques such as incorporating shallow cross-feature layers. The following pseudo-code illustrates this concept:

# Pseudo-code: A two-tower variant with an added feature cross layer
class EnhancedTwoTower(nn.Module):
    def __init__(self, user_feat_dim, item_feat_dim, emb_dim):
        super().__init__()
        self.user_tower = MLP(user_feat_dim, emb_dim)
        self.item_tower = MLP(item_feat_dim, emb_dim)
        # Introduce a shallow feature cross layer (e.g., FM/DeepFM idea)
        self.cross_layer = FeatureCrossLayer(user_feat_dim, item_feat_dim)
    def forward(self, user_feats, item_feats):
        # Basic two-tower dot product
        user_emb = self.user_tower(user_feats)
        item_emb = self.item_tower(item_feats)
        dot_score = torch.sum(user_emb * item_emb, dim=1)
        # Shallow cross-feature scoring (only for limited cross-features available online)
        cross_score = self.cross_layer(user_feats, item_feats)
        return dot_score + cross_score

Note: In practice, if such an online cross layer is introduced, it is crucial to ensure that the item features passed are static features that do not depend on user behavior during offline pre-computation. Otherwise, the premise for vector retrieval pre-computation is broken.

Advancing Beyond the Two-Tower Bottleneck: Introducing Feature Crossing

To improve the upper bound of coarse ranking performance and bring it closer to precise ranking, the industry has begun exploring paths that introduce complex crossing models while keeping computational costs manageable.

1. Downscaling Complex Networks from Precise Ranking

A common approach is to adopt models validated in precise ranking, such as DeepFM or DCN, but compress them by significantly reducing the number of features, lowering hidden layer dimensions, and shortening network depth. While this approach enhances model expressiveness, it forfeits the capability for vector pre-computation, requiring the system to have extremely high concurrent processing power. This often depends on high-performance computing platforms and elastic scaling mechanisms at the infrastructure level.

2. Incorporating User Behavior Sequence Modeling

In modern recommendation coarse ranking, a user’s long-term and short-term historical behavior sequences are crucial for determining interest. To introduce sequence features within a two-tower framework, target attention mechanisms are typically employed. This means that during online computation, the user-side tower dynamically retrieves and weights the user’s historical behavior sequence based on the current candidate item.

This practice deviates from strict offline pre-computation. However, a balance between computational cost and expressive effect can be achieved by pre-computing the embeddings for user behavior sequences and placing them in a high-performance cache. Online, only lightweight attention weighting computations are required.

Coarse Ranking Engineering Architecture and Cloud-Native Infrastructure

The evolution of coarse ranking models is not only a contest of algorithms but also a significant challenge for the underlying engineering architecture. When transitioning from a simple two-tower model to those with complex feature crossing and sequence modeling, single-machine memory and computational power often prove insufficient.

1. Feature Serving and High-Performance Cache: Coarse ranking relies heavily on a large number of statistical features and pre-computed vectors, typically read with low latency from distributed in-memory databases or generic cache services. For requirements on the order of a hundred milliseconds, feature read latency must be controlled to single-digit milliseconds.
2. Elastic Compute and Containerized Deployment: Coarse ranking models with feature crossing primarily depend on compute-intensive GPU or heterogeneous computing resources for online inference. During traffic peaks, the system must leverage cloud-native container platforms, such as managed Kubernetes services, to achieve automatic model scaling and ensure throughput is not compromised.
3. Logging and Monitoring Alert Systems: As a pivotal hub in the pipeline, a slight shift in the score distribution of coarse ranking can directly impact the quality of the candidate pool for precise ranking. Using generic logging services to collect coarse ranking score logs, combined with monitoring and alerting systems to track metrics like AUC and average scores in real-time, is essential for maintaining system stability.

Summary and Outlook

The design of a coarse ranking model is essentially a careful trade-off between “computational cost” and “effectiveness.” The two-tower structure, represented by DSSM, established the engineering baseline for coarse ranking by decoupling computations to achieve industrial-grade fast serving. The subsequent introduction of lightweight feature crossing and sequence modeling is a natural evolution aimed at overcoming its expressive limitations.

Looking ahead, as underlying heterogeneous computing power improves and cloud-native infrastructure matures, the boundaries between coarse ranking and precise ranking may gradually blur. End-to-end architectures based on joint training and dynamic routing hold the potential to further simplify the system pipeline, leading to new leaps in efficiency for search and recommendation systems. For developers and enterprises, building a robust infrastructure that supports elastic compute, comprehensive feature caching, and a mature monitoring system is fundamental to the continuous iteration and success of coarse ranking models.