Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval

Jul 16, 2026

Two-tower models are widely used in the retrieval stage of large-scale recommendation systems, yet standard in-batch and out-of-batch negative sampling often generates easy negatives and leaves models under-challenged. The paper proposes a self-supervised hard negative sampling method that uses an LLM-driven clustering to generate negatives from the same cluster during training. A real-time, low-complexity sampling framework enables billion-scale training integration. Experiments and industrial deployment show improved retrieval performance, higher CTR, and mitigation of popularity bias via feedback-loop breaking.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

Introduction

Related Work

Multi-stage System

arXiv :2607 .00448v2 [ cs .IR] 6 Jul 2026 Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval Ivan Ji 1 ,∗ , Liuyi Hu 1 ,∗ , Harrison (Zihao) Zhao 1 ,∗ , Lei Huang 1 , Qunshu Zhang 1 , Max (Xiangjun) Fan 1 , Aameek Singh 1 1 Meta ∗ Three authors contributed equally to this research. The two-tower model has been widely used for large-scale recommendation systems, particularly in the retrieval stage. Industry standards for training two-tower models typically involve in-batch and/or out-of-batch negative sampling. However, these methods often produce easy negatives that models can quickly learn, failing to sufficiently challenge the model. To address this issue, a novel self-supervised hard negative sampling technique is proposed that leverages a large language model (LLM) to generate hard negatives from the same cluster during model training. By utilizing the LLM to learn media representations, the proposed approach ensures that the generated negatives are more challenging and informative. This real-time sampling framework is designed for seamless integration into production models, capable of handling billions of training data points with minimal computational complexity. Experiments on public datasets, along with deployment to a large-scale online system, demonstrate that the proposed negative sampling technique outperforms widely used industry methods. Furthermore, analysis in industrial applications reveals that this sampling method can help break inherent feedback loops in recommendations and significantly reduce popularity bias. Date: July 7, 2026 Correspondence: Harrison (Zihao) Zhao [at](at harrisonzhao@meta.com)[ harrisonzhao@meta.com](at harrisonzhao@meta.com) 1 Introduction In the era of big data, large-scale recommendation systems have become increasingly important in various applications, including e-commerce, social media, and entertainment. These systems aim to provide users with personalized recommendations that cater to their interests and preferences. However, designing an efficient and effective recommendation system is a challenging task due to the vast number of candidates eligible for selection. To address this challenge, modern recommendation systems typically adopt a multi-stage design, consisting of retrieval and ranking stages Covington et al. (2016); Liu et al. (2017); Chen et al. (2017) . The retrieval stage aims to retrieve a small subset of relevant candidates from a large corpus, while the ranking stages further refine the selection to provide the most relevant recommendations. The two-tower model design is widely used as retrieval models in large scale applications due to its serving efficiency Covington et al.(2016); Huang et al. (2020) . Retrieval model training is often formulated as an extreme classification problem where negative samples are playing a critical role. Various sampling-based techniques have been proposed to enhance training efficiency Bengio and Senécal (2008, 2003); Covington et al. (2016) . The widely used sampling techniques are in-batch negative sampling Hidasi (2015); Gillick et al. (2019) and out-of-batch (OOB) negative sampling and also mixed negative sampling Yang et al. (2020); Hidasi and Karatzoglou (2018) . However, in-batch negatives are constrained by the mini-batch size, which can result in models experiencing recommendation bias and limited exposure to diverse corpus during training. And OOB negative samples are too easy for model to learn especially when the OOB item pool is very large and diverse. Besides, retrieval models typically use user engagement such as click as positives. However, whether user clicks or not depends on what item we surface to the user which is controlled by the multi-stage system. The strong feedback loop can result in popularity bias in the recommendation Chen et al. (2020); Cañamares and Castells (2018); Morik et al. (2020); Oosterhuis (2021) . To solve the problems of ne

FAQs

What problem does the paper address in two-tower retrieval training?

The paper targets two key issues: negative sampling often produces overly easy negatives, and feedback loops from engagement-based positives can introduce popularity bias.

How does the proposed hard negative sampling method work?

It leverages item clusters learned via a large language model to generate hard negatives on the fly from the same cluster during training.

What evidence supports the approach’s effectiveness and scalability?

Experiments on public datasets and deployment to a large-scale online system show that the method outperforms common industry negative sampling and improves CTR, while also reducing popularity bias.

Jul 16, 2026

Upload by Lucas Martin