RPDR - Round-trip Prediction-Based Data Augmentation Framework - for Long-Tail Question Answering

Oct 22, 2026

Uploaded by Sarah

Long-tail question answering challenges large language models due to limited acquisition and reliable recall of rare knowledge. Retrieval-augmented generation mitigates this by using external retrieval, yet dense retrievers can generalize poorly to rare or niche queries. RPDR introduces a data augmentation approach that selects high-quality, easy-to-learn training data to strengthen dense retrievers. It combines synthetic generation, round-trip selection, and retriever training, improving POPQA and ENTITY QUESTIONS over BM25 and Contriver.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Abstract

Introduction

Long-tail challenges and negative feedback loop

Retrieval-augmented generation and dense retrieval limits

RPDR Method

Synthetic data generation

Round-trip prediction-based data selection

Retriever training with augmented data

Experiments

POPQA and ENTITY QUESTIONS results

Impact on long-tail accuracy

RPDR: A Round-trip Prediction-Based Data Augmentation Framework for Long-Tail Question Answering Yiming Zhang 1 * Siyue Zhang2 Junbo Zhao 1 Chen Zhao3 ,4 1Zhejiang University 2Nanyang Technological University 3 NYU Shanghai 4 Center for Data Science, New York University arXiv :2602 . 17366v1 [ cs .CL] 19 Feb 2026 Abstract Long-tail question answering presents significant challenges for large language models (LLMs) due to their limited ability to acquire and accurately recall less common knowledge. Retrieval-augmented generation (RAG) systems have shown great promise in mitigating this limitation by integrating external retrieval mechanisms. However, dense retrieval models often face the same difﬁculties when generalizing to rare or niche knowledge. In this study, we introduce RPDR, a novel data augmentation framework that selects high-quality easy-to-learn training data, to enhance dense retrievers. Our approach is built around three core components: synthetic data generation, data selection with RoundTrip prediction to identify easy-to-learn instances, and retriever training with these instances. We evaluate RPDR on two longtail retrieval benchmarks, POPQA and ENTITY QUESTIONS, demonstrating substantial improvements over existing retrievers like BM25 and Contriver, especially on extremely longtail categories. We identify the strengths and limitations of RPDR through detailed human analysis and propose a dynamic routing mechanism to dynamically route queries to specialized retrieval modules to further improve retrieval performance.1 1 Introduction The development of large language models (LLMs) has transformed a wide range of applications, from answering questions to supporting complex decision-making (Wu et al., 2024) . Despite their impressive capabilities, a key challenge arises when generalizing these models to realworld scenarios: handling long-tail user queries *Work done when Yiming Zhang was visiting NYU Shanghai. Correspondence: Yiming Zhang ([yimingz@zju.edu.cn](yimingz@zju.edu.cn)), Chen Zhao ( [cz1285@nyu.edu](cz1285@nyu.edu)) 1Our code will be made available at [https://](https://)[ ](https://)[github.com/yiming-zh/RPDR](github.com/yiming-zh/RPDR). Retr ieva l Accuracy 1.0 0.8 0.6 0.4 0.2 0.0 | | | | | | --- | --- | --- | --- | | | | | | Long-tail Infrequent Frequent Figure 1: (Top) Negative feedback loop in long-tail question answering in large language model-based systems. (Bottom) We challenge existing ﬁndings that dense retrievers struggle on long-tail questions, and argue that through appropriate training, dense retrieval based methods RPDR can surpass BM25 on long-tail retrieval. (Kandpal et al., 2023 ; Dai et al., 2023) . For instance, a user might ask about a lesser-known football player as shown in Figure 1. Addressing longtail question answering (LTQA) is critical because it directly inﬂuences user trust and engagement. When systems fail to provide accurate answers to these uncommon queries, users may lose conﬁdence in the system, reducing their likelihood of asking similar questions in the future. This interaction creates a negative feedback loop resulting in a reduction in long-tail training data that further degrades the model’s performance. Given the scale of todays pre-training data and LLMs, it is expected that LLMs can acquire a huge amount of information and answer longtail questions with their parametric knowledge. However, recent work (Kandpal et al., 2023) has shown that LLMs struggle to learn long-tail knowledge, and are prone to hallucinating in such cases. Retrieval-augmented generation (RAG) has emerged as a promising solution for LTQA (Wu et al., 2024) . By integrating external knowledge through retrieval mechanisms, RAG systems aim to mitigate the limitations of relying solely on the parametric knowledge of LLMs. Despite their high potential, the technical challenge is how to retrieve relevant knowledge from millions of candidates for

FAQs

What problem does RPDR address in long-tail question answering?

RPDR targets the difficulty large language models and dense retrievers have when generalizing to rare or niche knowledge in long-tail queries.

How does RPDR select training data for improving dense retrievers?

RPDR uses round-trip prediction: an off-the-shelf dense retriever produces embeddings, and a trained inverse model reconstructs the instance, selecting easy-to-learn question-answer pairs.

What datasets and baselines are used to evaluate RPDR?

RPDR is evaluated on POPQA and ENTITY QUESTIONS, showing improvements over BM25 and dense retrievers such as Contriver, especially on extremely long-tail categories.

Oct 22, 2026

Uploaded by Sarah