[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-147253-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-147253-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","how-to-train-your-dragon-diverse-augmentation-towards-generalizable-dense-retrieval","How to Train Your DRAGON - Diverse Augmentation Towards Generalizable Dense Retrieval","","Dense retrieval (DR) techniques aim to improve ranking quality, yet existing approaches often face a tradeoff between supervised and zero-shot performance, commonly attributed to limited model capacity. This paper challenges that assumption by showing a generalizable DR can reach strong accuracy in both settings without increasing model size. It analyzes contrastive learning under data augmentation and finds typical augmentation and pseudo-labeling strategies inefficient and suboptimal. The proposed DRAGON method uses diverse queries and supervision to progressively train an effective BERT-base-sized retriever achieving state-of-the-art results.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/how-to-train-your-dragon-diverse-augmentation-towards-generalizable-dense-retrieval/147253/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/how-to-train-your-dragon-diverse-augmentation-towards-generalizable-dense-retrieval/147253.png","ImageObject",300,407,{"name":42,"@type":43},"Ophelia","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-17","2026-08-26",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",14,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does this paper address in dense retrieval?","Question",{"text":63,"@type":64},"The paper addresses the effectiveness tradeoff between supervised and zero-shot dense retrieval, which many believe comes from limited model capacity.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What main hypothesis does the paper challenge?",{"text":68,"@type":64},"It contradicts the hypothesis that the tradeoff is unavoidable due to insufficient capacity, arguing that generalizable dense retrieval can be trained without increasing model size.",{"name":70,"@type":61,"acceptedAnswer":71},"How does DRAGON improve dense retrieval training?",{"text":72,"@type":64},"DRAGON proposes a new data augmentation approach using diverse queries and multiple sources of supervision to progressively train a generalizable retriever.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},147253,1787760839,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":109,"preview_page_count":4},7971461741311,"https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826","How to Train Your DRAGON:  \nDiverse Augmentation Towards Generalizable Dense Retrieval  \nSheng-Chieh Lin 1∗, Akari Asai2 , Minghan Li 1 , Barlas Oguz3 , Jimmy Lin 1 , Yashar Mehdad3 , Wen-tau Yih3 , and Xilun Chen3†  \nUniversity of Waterloo 1 , University of Washington2 , Meta AI3  \n{s269lin,m692li, [jimmylin}@uwaterloo.ca](jimmylin}@uwaterloo.ca), [akari@cs.washington.edu](akari@cs.washington.edu)[ ](akari@cs.washington.edu){barlaso,mehdad,scottyih,[xilun}@meta.com](xilun}@meta.com)  \nAbstract  \nVarious techniques have been developed in recent years to improve dense retrieval (DR), such as unsupervised contrastive learning and pseudo-query generation. Existing DRs, however, often suffer from effectiveness tradeoffs between supervised and zero-shot retrieval, which some argue was due to the limited model capacity. We contradict this hypothesis and show that a generalizable DR can be trained to achieve high accuracy in both supervised and zero-shot retrieval without increasing model size. In particular, we systematically examine the contrastive learning of DRs, under the framework of Data Augmentation (DA) . Our study shows that common DA practices such as query augmentation with generative models and pseudo-relevance label creation using a cross-encoder, are often inefficient and suboptimal. We hence propose a new DA approach with diverse queries and sources of supervision to progressively train a generalizable DR. As a result, DRAGON, 1 our Dense Retriever trained with diverse AuGmentatiON, is the first BERTbase-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations and even competes with models using more complex late interaction.  \n1 Introduction  \nBi-encoder based neural retrievers allow documents to be pre-computed independently of queries and stored, enabling end-to-end retrieval among huge corpus for downstream knowledgeintensive tasks (Karpukhin et al., 2020 ; Reimersand Gurevych, 2019) . Recently, Thakur et al.(2021b) show that bi-encoder retrievers still underperform BM25 in real-world scenarios, where training data is scarce. One potential solution is  \n∗ This work is done during Sheng-Chieh’s internship at Meta.† Xilun and Sheng-Chieh contributed equally to this work.  \n1The code and model checkpoints are available at: [https:](https:)//[github.com/facebookresearch/dpr-scale](github.com/facebookresearch/dpr-scale)  \nZero-shot evaluation (BEIR 13 subsets: nDCG@10)  \n52  \n50  \n48  \n46  \n44  \n42  \n40  \n38  \n\n|  |  |  |  |  | SPLADE++\u003Cbr>\u003Cbr>| ColBERTv2 |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | \u003Cbr>C\u003Cbr>\u003Cbr>RetroM | OCO-DR\u003Cbr>AE |  |  | GTR-XXL\u003Cbr>| \u003Cbr>\u003Cbr>DRAGON |\n|  Contriever | supervised and | zero | h |  |  |  |\n|  |  | -s ot effectiv |  | eness tradeoff | \u003Cbr>CL-DRD |  |\n|  Dense r | etrieval |  |  | coCo | ndenser\u003Cbr> R\u003Cbr>| ocketQAv2 |\n| \u003Cbr>Dense r\u003Cbr>(40X lar | etrieval ger model) etrieval |  |  |  |  | COT-MAE |\n|   Sparse r\u003Cbr> Multi-ve | ctor retrieval |  |  |  |  | |\n\n34 35 36 37 38 39 40  \nSupervised evaluation (MS MARCO Dev: RR@10)  \nFigure 1: Supervised versus zero-shot effectiveness comparison among existing state-of-the-art retrievers. All models use a BERT-base-sized (110M parameters) backbone except for GTR-XXL (4.8B parameters) .  \nto design more expressive representations to capture more fine-grained token-level information; e.g., SPLADE++ (Formal et al., 2022) and ColBERTv2 (Santhanam et al., 2022b) in Figure 1. However, these designs add complexity and latency to retrieval systems (Mackenzie et al., 2021) .  \nBy contrast, dense retrieval (DR) is a simpler biencoder retriever that maps queries and documents into low-dimensional vectors and computes text similarity through a simple dot product. Top-k retrieval can be performed directly using ANN search libraries (Johnson et al., 2021 ; Guo et al., 2020) . Recently, various methods have been proposed to improve DR effectiveness while keeping its simple architecture, such as pre-training (Lee et al","cbCaiaAY34OdCwmC","https://ap.wps.com/l/cbCaiaAY34OdCwmC","pdf",590287,16,"English","# Abstract\n# Introduction","[{\"question\":\"What problem does this paper address in dense retrieval?\",\"answer\":\"The paper addresses the effectiveness tradeoff between supervised and zero-shot dense retrieval, which many believe comes from limited model capacity.\"},{\"question\":\"What main hypothesis does the paper challenge?\",\"answer\":\"It contradicts the hypothesis that the tradeoff is unavoidable due to insufficient capacity, arguing that generalizable dense retrieval can be trained without increasing model size.\"},{\"question\":\"How does DRAGON improve dense retrieval training?\",\"answer\":\"DRAGON proposes a new data augmentation approach using diverse queries and multiple sources of supervision to progressively train a generalizable retriever.\"}]","How to Train Your DRAGON - Diverse Augmentation Towards Generalizable Dense Retrieval | PDF"]