[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82126-en":3,"doc-seo-82126-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82126,1099514067415,"Rowan","https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502",8,"Research & Report","Optimal Top k Identification from Pairwise Comparisons","Active learning fixed-confidence top-k identification from noisy pairwise comparisons is studied. An adaptive algorithm selects item pairs sequentially, observes noisy outcomes, and stops once it can output the true top-k set with error probability at most δ. The goal is a δ-correct procedure that minimizes expected comparisons, i.e., sample complexity. The work develops a computationally efficient approach by characterizing lower-bound structure as a saddle-point problem and proving asymptotic optimality of an adaptive allocation strategy.","Optimal Top-k Identification from Pairwise Comparisons  \nMotti Goldberger 1 Nils Rudi 1  \narXiv :2607 .08979v 1 [ cs .LG] 9 Jul 2026  \nAbstract  \nWe study the active learning problem of fixedconfidence top-k identification from noisy pairwise comparisons. In this problem, an algorithm sequentially chooses pairs of items to compare, observes the outcomes, and stops when it can return the set of top-k items with error probability at most δ . The objective is to design such a δ -correct procedure that minimizes the expected number of comparisons (the sample complexity) .  \nThis problem falls within the broader literature on fixed-confidence pure exploration in bandit models, where a common target is asymptotic optimality: the algorithm’s expected sample complexity matches the information theoretic lower bound as δ → 0. Asymptotically optimal procedures have been developed for a range of fixed-confidence pure-exploration problems, however to the best of our knowledge, for top-1, or more generally top-k identification from pairwise comparisons under latent utility models an asymptotically optimal algorithm has not been established. In this setting, we develop such an algorithm. We characterize the structure of the lower bound and formulate itas a saddle-point problem. This structure enables a computationally efficient primal–dual procedure that learns the asymptotically optimal comparison allocation online. We then construct an adaptive comparison-allocation algorithm that tracks the allocation learned by the primal–dual procedure and prove it is asymptotically optimal.  \n1. Introduction  \nThe objective of identifying the top-k items from a set of candidates using noisy pairwise comparisons arises in many settings. Consider a practitioner who wants to determine which large language model is best for their task. They open  \n1Yale University, New Haven, CT, USA. Correspondence to: Motti Goldberger \u003C[motti.goldberger@yale.edu](motti.goldberger@yale.edu) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \nArena (Chiang et al., 2024) (formerly Chatbot Arena), a public platform for evaluating LLMs through crowdsourced pairwise human preferences, and scan the leaderboard. The leaderboard ranks all relevant models, but the practitioner only wants to consider the top few on the list.  \nAdditional settings that use pairwise comparisons for top-k identification include: crowdsourced pairwise comparisons to identify the best photographs, translations, or annotations (Narimanzadeh et al., 2023; Kou et al., 2017; Chen et al., 2013); recommender systems that elicit pairwise feedback to identify a small set of items to display to a user (Kalloori et al., 2018); and sports tournaments.  \nTop-k identification serves as the natural objective in two distinct settings. In the first setting, the top-k set is the desired output itself. For example, the single best LLMis selected (k = 1), a funding program supports the top five proposals, or a streaming service displays a top ten list. In the second setting, top-k identification is the first stage of a two-stage process, with the second stage using a different and/or more expensive signal. For example, Arena could be used to narrow the field of candidate models to k , after which a more careful application-specific evaluation is conducted on the finalists. The top-k identification in the first stage can be essential when the time and resources required for a second stage with all items is prohibitively large.  \nIn both settings, comparisons cost money, time, and/or scarce human attention, which makes adaptivity valuable. Rather than fixing a static sampling plan up front, an experiment designer can adaptively choose which pair to compare next based on outcomes observed so far, allocating effort on comparisons that are most informative for distinguishing the top-k items from the rest. We study this problem in t","cbCaiieKokCIygfS","https://ap.wps.com/l/cbCaiieKokCIygfS","pdf",1943817,1,28,"English","en",105,"# Abstract\n# Introduction\n## Problem setting and objective\n## Modeling assumptions for pairwise comparisons\n## Latent-utility and cardinal outcomes\n## Fixed-confidence identification and lower bounds","[{\"question\":\"What is the goal of the top-k identification problem studied here?\",\"answer\":\"The goal is to identify the set of top-k items from noisy pairwise comparisons while controlling the error probability to be at most δ.\"},{\"question\":\"How does the algorithm decide which pair of items to compare next?\",\"answer\":\"It adaptively chooses item pairs sequentially based on observed outcomes so far, aiming to allocate comparisons most effectively to distinguish the top-k items from the rest.\"},{\"question\":\"What does δ-correct and asymptotically optimal mean in this context?\",\"answer\":\"δ-correct means stopping only when the returned top-k set has error probability at most δ. Asymptotically optimal means the expected sample complexity matches the information-theoretic lower bound as δ → 0.\"}]",1784178347,71,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"optimal-top-k-identification-from-pairwise-comparisons","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/optimal-top-k-identification-from-pairwise-comparisons/82126/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What is the goal of the top-k identification problem studied here?","Question",{"text":74,"@type":75},"The goal is to identify the set of top-k items from noisy pairwise comparisons while controlling the error probability to be at most δ.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the algorithm decide which pair of items to compare next?",{"text":79,"@type":75},"It adaptively chooses item pairs sequentially based on observed outcomes so far, aiming to allocate comparisons most effectively to distinguish the top-k items from the rest.",{"name":81,"@type":72,"acceptedAnswer":82},"What does δ-correct and asymptotically optimal mean in this context?",{"text":83,"@type":75},"δ-correct means stopping only when the returned top-k set has error probability at most δ. Asymptotically optimal means the expected sample complexity matches the information-theoretic lower bound as δ → 0.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]