[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85691-en":3,"doc-seo-85691-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85691,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks","The study investigates LLM benchmark coreset selection by choosing a small subset of prompts so that induced model scores and rankings closely match those from the full benchmark suite. It focuses on evaluation-unsupervised coreset selection, where no model evaluation outcomes are used and subsets are constructed at prompt-level across multiple benchmarks. Using submodular selection, it compares DPP-based, submodular mutual information, and facility-location functions. On 35 heterogeneous benchmarks with 18 frontier LLMs and 61K+ prompts, facility location on semantic embeddings best preserves scores across budgets.","Coresets Before Score Sets:  \nEvaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks  \nJihan Yao1 * , Gantavya Bhatt1 * , Arnav Das1 * , Peter Jin6 , Ke Bao5 , Qiaolin Yu5 ,  \nKhushi Bhardwaj6 , Chang Su3 , Jialei Wang6 , Yikai Zhu5 , Sugam Devare6 , Damon Mosk-Aoyama6 , Zhen Dong6 , Venkat Krishna Srinivasan6 , Yineng Zhang4 ,  \nOleksii Kuchaiev6 , Jiantao Jiao2,6 , Banghua Zhu1,5 , Jeff Bilmes1  \n1University of Washington, Seattle 2University of California, Berkeley  \n3 Oracle 4Together AI 5LMSYS 6NVIDIA  \nCorrespondence: [jihany2@cs.washington.edu](jihany2@cs.washington.edu), [gbhatt2@uw.edu](gbhatt2@uw.edu), [arnavmd2@uw.edu](arnavmd2@uw.edu)  \narXiv :2607 .09739v 1 [ cs .AI] 2 Jul 2026  \nAbstract  \nWe study LLM benchmark coreset selection: selecting a small subset of prompts over multiple benchmarks whose induced model scores and rankings approximate those obtained from the full benchmark suite. In evaluation-unsupervised benchmark coreset selection (our approach), the selection algorithm uses no model evaluation outcomes, and operates on a fine granularity by producing subsets of prompts over multiple benchmarks rather than producing a sub-collection of entire benchmarks. We use submodular subset selection, and we develop and evaluate many different submodular functions for this purpose, including determinantal point process (DPP) based approaches, submodular mutual information functions, and facility location-based functions. On a new large-scale suite of 35 heterogeneous benchmarks spanning five different capability categories, 18 frontier LLMs, and over 61K prompts, we find that the facility location (FL) function operating exclusively on inexpensive semantic prompt embeddings preserves LLM scores better than twelve separate score-based and diversity-based baselines, across a range of coreset budgets. Moreover, we show our proposed objective is not limited to the evaluationunsupervised regime: in the setting where only a handful of whole benchmarks must be selected and a large amount of model scores are available, the same objective matches or outperforms state-of-the-art baselines on the MMLU and MTEB leaderboards, while being substantially cheaper to compute. Together, our results suggest that submodularity, in general, is a strong and reliable tool for benchmark compression.  \n1 Introduction  \nLLM evaluation is a major but necessary cost in large language model (LLM) development. As  \n*Equal contribution, random order.  \nmodels are deployed across increasingly specialized settings, evaluation suites have expanded from a small number of general-purpose benchmarksto large collections spanning knowledge, mathematics, coding, instruction following, and agentic behavior (Phan et al., 2025 ; Wang et al., 2024 ; Merrill et al., 2026 ; Pyatkin et al., 2026 ; Yao et al., 2024) . Typically, LLM evaluation is repeatedly performed throughout the full LLM development cycle as new models are regularly created. However, repeatedly running evaluations on the full suite of benchmarks is computationally expensive and costly. Recently, a large body of work has shown that many benchmarks contain substantial redundancy (DatologyAI Team, 2026 ; Polo et al., 2024), meaning that benchmark evaluation is naturally inefficient, since different parts of different benchmarks might redundantly evaluate the same model ability (also see Figure 6) . This has motivated a growing focus on benchmark coresets 1 : i.e., selecting a much smaller set of examples or benchmarks (or portions thereof) that preserves the evaluation results of a full evaluation suite (Viveket al., 2024 ; Kipnis et al., 2025) .  \nIn this paper we study LLM benchmark coreset selection: selecting a small subset of benchmark prompts whose induced model scores and rankings approximate those obtained from the complete benchmark suite. We distinguish between two variants of this problem.  \nIn evaluation-supervised benchmark coreset selection2 , the coreset algorithm may u","cbCairWujTtlEJRr","https://ap.wps.com/l/cbCairWujTtlEJRr","pdf",712091,1,21,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What is the main goal of LLM benchmark coreset selection in this work?\",\"answer\":\"Select a small subset of prompts whose induced model scores and rankings approximate the results from evaluating the full benchmark suite.\"},{\"question\":\"How does the evaluation-unsupervised setting differ from evaluation-supervised coreset selection?\",\"answer\":\"Evaluation-unsupervised selection uses no model evaluation outcomes, while evaluation-supervised selection can rely on prior model-by-prompt evaluation results as supervision.\"},{\"question\":\"Which coreset objective performs best on the large-scale benchmark suite?\",\"answer\":\"A facility location (FL) function operating solely on inexpensive semantic prompt embeddings preserves LLM scores better than multiple score-based and diversity-based baselines across coreset budgets.\"}]",1784205622,53,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"coresets-before-score-sets-evaluation-unsupervised-prompt-subset-selection-for-llm-benchmarks","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/coresets-before-score-sets-evaluation-unsupervised-prompt-subset-selection-for-llm-benchmarks/85691/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main goal of LLM benchmark coreset selection in this work?","Question",{"text":75,"@type":76},"Select a small subset of prompts whose induced model scores and rankings approximate the results from evaluating the full benchmark suite.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the evaluation-unsupervised setting differ from evaluation-supervised coreset selection?",{"text":80,"@type":76},"Evaluation-unsupervised selection uses no model evaluation outcomes, while evaluation-supervised selection can rely on prior model-by-prompt evaluation results as supervision.",{"name":82,"@type":73,"acceptedAnswer":83},"Which coreset objective performs best on the large-scale benchmark suite?",{"text":84,"@type":76},"A facility location (FL) function operating solely on inexpensive semantic prompt embeddings preserves LLM scores better than multiple score-based and diversity-based baselines across coreset budgets.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]