[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-503299-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-503299-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","explora-efficient-exemplar-subset-selection-for-complex-reasoning","EXPLORA - Efficient Exemplar Subset Selection for Complex Reasoning","","Complex, multi-step question answering over structured and unstructured sources remains difficult, especially when using in-context learning (ICL) with large language models. Selecting optimal exemplars is a core bottleneck, since exemplar subsets may be task-specific (static) or test-example-specific (dynamic). This paper introduces EXPLORA, a static exemplar subset selection method that explores subsets to estimate a scoring function without confidence signals. Experiments show a 12.24% improvement while reducing LLM calls to about 11% of prior state of the art.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/explora-efficient-exemplar-subset-selection-for-complex-reasoning/503299/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/explora-efficient-exemplar-subset-selection-for-complex-reasoning/503299.png","ImageObject",300,407,{"name":42,"@type":43},"Damian","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-16","2026-10-03",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",5,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does EXPLORA address in in-context learning?","Question",{"text":63,"@type":64},"EXPLORA addresses the selection of optimal exemplars for complex reasoning tasks, focusing on efficient static exemplar subset selection rather than choosing exemplars at query time.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does EXPLORA estimate the quality of exemplar subsets?",{"text":68,"@type":64},"EXPLORA uses exploration to estimate parameters of a scoring function that evaluates exemplar subsets without incorporating confidence information, using a loss model based on similarity and validation examples.",{"name":70,"@type":61,"acceptedAnswer":71},"What improvements does the paper report compared with existing methods?",{"text":72,"@type":64},"EXPLORA outperforms static and dynamic exemplar selection baselines by 12.24% and 45.45%, respectively, while reducing the number of LLM calls to roughly 11% of the state of the art.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},503299,1791170296,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,100,105,110,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":97,"show_sort_weight":98,"slug":99},"Comic",60,"comic",{"id":101,"doc_module":4,"doc_module_name":25,"category_name":102,"show_sort_weight":103,"slug":104},6,"Technology",50,"technology",{"id":106,"doc_module":4,"doc_module_name":25,"category_name":107,"show_sort_weight":108,"slug":109},7,"Healthcare",40,"healthcare",{"id":111,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},8,30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":55,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":111,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145},137451208677,"https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d","EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning  \nKiran Purohit  \nIIT Kharagpurkiran.purohit @ [kgpian.iitkgp.ac.in](kgpian.iitkgp.ac.in)  \nVenktesh V  \nTU Delft  \nv .viswanathan-1 @[tudelft.nl](tudelft.nl)  \nRaghuram Devalla  \nIIT Kharagpur devallaraghuram @ [gmail.com](gmail.com)  \nKrishna Mohan Yerragorla  \nIIT Kharagpur krishnamohanyerragorla  \n@ [gmail.com](gmail.com)  \nSourangshu Bhattacharya  \nIIT Kharagpur  \nsourangshu @ [cse.iitkgp.ac.in](cse.iitkgp.ac.in)  \nAvishek Anand  \nTU Delft  \navishek .anand @[tudelft.nl](tudelft.nl)  \nAbstract  \nAnswering reasoning-based complex questions over text and hybrid sources, including tables, is a challenging task. Recent advances in large language models (LLMs) have enabled in-context learning (ICL), allowing LLMs to acquire proficiency in a specific task using only a few demonstration samples (exemplars) . A critical challenge in ICL is the selection of optimal exemplars, which can be either taskspecific (static) or test-example-specific (dynamic) . Static exemplars provide faster inference times and increased robustness across a distribution of test examples. In this paper, we propose an algorithm for static exemplar subset selection for complex reasoning tasks. We introduce EXPLORA, a novel exploration method designed to estimate the parameters of the scoring function, which evaluates exemplar subsets without incorporating confidence information. EXPLORA significantly reduces the number of LLM calls to ∼ 11% of those required by state-of-the-art methods and achieves a substantial performance improvement of 12.24% . We open-source our code and data 1.  \n1 Introduction  \nAnswering complex questions that require multistep reasoning (Chen et al., 2022b ; Lu et al., 2023a ; Ling et al., 2017 ; Roy and Roth, 2016 ; Roy and Anand, 2022 ; Venktesh et al., 2024) over structured and unstructured sources is an active research area with applications in finance, law, fact-checking and healthcare (Wang et al., 2023a ; Zhang et al., 2021) . Unlike fine-tuning task-specific models (Chiang and Chen, 2019 ; Amini et al., 2019 ; Ling et al., 2017 ; Roy and Roth, 2016 ; Geva et al., 2020 ; Cobbe et al., 2021), recent advances in Large Language Models (LLMs) have paved the way for new approaches that employ in-context learning (ICL)  \n1 [https://github.com/kiranpurohit/](https://github.com/kiranpurohit/)[ ](https://github.com/kiranpurohit/)EXPLORA  \n(Wei et al., 2023) to solve complex reasoning problems. This approach focuses on choosing a small number of demonstration examples to be used as input prompts to LLMs.  \nAn effective way to tackle complex reasoning problems is to use chain-of-thought (COT) prompting, which adds hand-crafted natural language rationales as stepwise solutions to the prompts, resulting in the triplet (input, rationale, output) (Wei et al., 2023) . In this work, we refer to this triplet as an exemplar. A limitation of the COT-based method for reasoning tasks is the tedious and non-scalable manual effort required in selecting the rationales or exemplars (Lu et al., 2022 ; Zhao et al., 2021 ; Chang and Jia, 2023) . To address these limitations, both static and dynamic approaches for automatic exemplar selection have been proposed (Ye et al., 2023a ; Lu et al., 2022 ; Rubin et al., 2022 ; Fu et al., 2023) . Some approaches require annotated data and training of multiple models for exemplar selection (Lu et al., 2023b ; Ye et al., 2023a) . Dynamic exemplar selection methods often involve additional computational costs because they select exemplars during query time, necessitating extensive query encoding and dynamic exploration of the search space. In contrast, static exemplar selection pre-selects a small subset of exemplars, which are used during LLM inference. Prior exemplar selection methods do not capture interactions between the exemplarsin the selected set (Li and Qiu, 2023) . Additionally, current static selection approaches (Li and Qiu, 2023) are chara","cbCaicJcVQgxA9qO","https://ap.wps.com/l/cbCaicJcVQgxA9qO","pdf",461892,22,"English","# Abstract\n# Introduction\n## In-context learning and exemplar selection\n## Chain-of-thought and exemplar triplets\n## Static vs dynamic exemplar selection\n## Proposed method and hypotheses","[{\"question\":\"What problem does EXPLORA address in in-context learning?\",\"answer\":\"EXPLORA addresses the selection of optimal exemplars for complex reasoning tasks, focusing on efficient static exemplar subset selection rather than choosing exemplars at query time.\"},{\"question\":\"How does EXPLORA estimate the quality of exemplar subsets?\",\"answer\":\"EXPLORA uses exploration to estimate parameters of a scoring function that evaluates exemplar subsets without incorporating confidence information, using a loss model based on similarity and validation examples.\"},{\"question\":\"What improvements does the paper report compared with existing methods?\",\"answer\":\"EXPLORA outperforms static and dynamic exemplar selection baselines by 12.24% and 45.45%, respectively, while reducing the number of LLM calls to roughly 11% of the state of the art.\"}]","EXPLORA - Efficient Exemplar Subset Selection for Complex Reasoning | PDF",1791053360,55]