[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119639-en":3,"doc-seo-119639-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119639,13056703020460,"Valentina","https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923",8,"Research & Report","Retrieval-Enhanced Machine Learning: Synthesis and Opportunities","Retrieval-Enhanced Machine Learning (REML) addresses key limitations in language modeling by integrating retrieval components to improve knowledge grounding, interpretability, and scalability. The work proposes a formal framework and synthesizes literature across multiple machine learning domains using consistent notations. It also identifies a gap: many retrieval-augmented studies lack integration with foundational Information Retrieval (IR) research. The goal is to provide researchers a structured foundation for building retrieval-enhanced models and enable interdisciplinary research.","arXiv :2407 . 12982v2 [ cs .LG] 18 Oct 2024  \nRetrieval-Enhanced Machine Learning: Synthesis and Opportunities  \nTO EUN KIM, Carnegie Mellon University, United States  \nALIREZA SALEMI, University of Massachusetts Amherst, United States ANDREW DROZDOV∗ , University of Massachusetts Amherst, United States FERNANDO DIAZ, Carnegie Mellon University, United States  \nHAMED ZAMANI, University of Massachusetts Amherst, United States  \nIn the field of language modeling, models augmented with retrieval components have emerged as a promising solution to address several challenges faced in the natural language processing (NLP) field, including knowledge grounding, interpretability, and scalability. Despite the primary focus on NLP, we posit that the paradigm of retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) such as computer vision, time series prediction, and computational biology. Therefore, this work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. Also, we found that while a number of studies employ retrieval components to augment their models, there is a lack of integration with foundational Information Retrieval (IR) research. We bridge this gap between the seminal IR research and contemporary REML studies by investigating each component that comprises the REML framework. Ultimately, the goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.  \nCCS Concepts: • Information systems → Information retrieval; • Computing methodologies → Machine learning.  \nAdditional Key Words and Phrases: Information Retrieval, Machine Learning  \n1 Introduction  \nBackground. In recent years, the research landscape surrounding large language models (LLMs) has witnessed substantial growth, underscored by the profound potential these models hold for various natural language processing (NLP) tasks. One of the significant advancements that has propelled this field forward is the scaling of the number of parameters of LLMs, which has enabled the training of models with unprecedented size and complexity [267] . We witness a similar trend in other fields adjacent to machine learning, for example, large vision foundation models for representing images and videos [5, 40] . Concurrently, the notion of in-context learning (ICL) [39] has emerged as a transformative capability, allowing LLMs to dynamically adapt and incorporate new information during its inference. In parallel, the information retrieval (IR) community has been actively exploring techniques aimed at improving the efficiency, effectiveness, and robustness of accessing information from large-scale collections. The convergence of these two domains has given rise to a new trend in research, where models are equipped with retrieval capabilities to access external knowledge during both training and inference stages [148, 257] . This integration of retrieval mechanisms into the prediction pipeline started to gain significant traction, as it allows models to ground their predictions in external knowledge without necessitating an increase in model capacity. Methods presented by Hashemi et al. [68] and Lewis et al.  \n[123] are among the earliest work in this space; the former focuses on retrieval-augmented representation learning by extending the transformer network, while the latter studies the paradigm of retrieval-augmented generation (RAG) for knowledge-intensive language tasks. That said, using retrieval results to improve a machine learning systems is not new. Pseudo-relevance feedback methods—methods for representing search queries using the top retrieved documents—are perhaps the first set of methods in this category [10, 30] . The ICL ability inherent i","cbCaijUxPeXIW2kc","https://ap.wps.com/l/cbCaijUxPeXIW2kc","pdf",1154226,1,30,"English","en",105,"# Introduction\n## Background\n## Motivation","[{\"question\":\"What problems does retrieval-enhanced machine learning aim to solve in language modeling?\",\"answer\":\"It targets knowledge grounding, interpretability, and scalability challenges in natural language processing by augmenting models with retrieval components.\"},{\"question\":\"How does the paper define Retrieval-Enhanced Machine Learning (REML)?\",\"answer\":\"It introduces a formal framework that synthesizes relevant literature across machine learning domains and aligns studies with consistent notations.\"},{\"question\":\"What gap does the paper identify between current REML work and Information Retrieval research?\",\"answer\":\"It finds that, although many studies use retrieval components, there is insufficient integration with foundational IR research.\"}]","Retrieval-Enhanced Machine Learning: Synthesis and Opportunities | PDF",1785725422,76,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"retrieval-enhanced-machine-learning-synthesis-and-opportunities","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/retrieval-enhanced-machine-learning-synthesis-and-opportunities/119639/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problems does retrieval-enhanced machine learning aim to solve in language modeling?","Question",{"text":75,"@type":76},"It targets knowledge grounding, interpretability, and scalability challenges in natural language processing by augmenting models with retrieval components.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper define Retrieval-Enhanced Machine Learning (REML)?",{"text":80,"@type":76},"It introduces a formal framework that synthesizes relevant literature across machine learning domains and aligns studies with consistent notations.",{"name":82,"@type":73,"acceptedAnswer":83},"What gap does the paper identify between current REML work and Information Retrieval research?",{"text":84,"@type":76},"It finds that, although many studies use retrieval components, there is insufficient integration with foundational IR research.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":21,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]