[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-504905-105":59,"doc-detail-504905-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","a-portfolio-based-analysis-method-for-competition-results-read-online-free","A portfolio-based analysis method for competition results - read online free","","A portfolio-based analysis method explains how to gain complementary insights from competition solvers beyond traditional rankings based on average performance. Motivated by algorithm portfolios and algorithm selection, the approach analyzes solver contribution from a portfolio viewpoint, addressing limitations of marginal-contribution metrics used in cooperative challenges. The method is demonstrated on MiniZinc Challenges results, highlighting how alternative measures such as Shapley values can produce different and more informative rankings.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/a-portfolio-based-analysis-method-for-competition-results-read-online-free/504905/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/a-portfolio-based-analysis-method-for-competition-results-read-online-free/504905.png","ImageObject",300,407,{"name":92,"@type":93},"Grenda","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-11","2026-10-04",true,{"@type":102,"interactionType":103,"userInteractionCount":24},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"Why is a traditional competition ranking based on average performance insufficient?","Question",{"text":112,"@type":113},"It may not reflect a solver’s full potential, since a solver can rank low overall while uniquely performing well on a subset of instances. This matters when leveraging portfolios rather than a single solver.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"How do cooperative competitions like Sparkle change how solver performance is assessed?",{"text":117,"@type":113},"They build an algorithm selector from participating solvers’ portfolio and rank solvers by marginal contribution to the selector’s performance on an instance set. This can credit solvers that look weak under official ranking.",{"name":119,"@type":110,"acceptedAnswer":120},"What issue can arise with marginal contribution measurement in portfolio settings?",{"text":121,"@type":113},"If two solvers jointly outperform others on a large subset but each is interchangeable, their marginal contributions can be zero, failing to reflect their true impact on overall portfolio performance.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},504905,1791320853,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":24,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":139,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":144,"read_time":145},7971474921005,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","arXiv :2205 . 15414v1 [ cs .AI] 30 May 2022  \nA portfolio-based analysis method for competition results  \nNguyen Dang !   \nUniversity of St Andrews, United Kingdom  \n~~ Abstract ~~  \nCompetitions such as the MiniZinc Challenges or the SAT competitions have been very useful sources for comparing performance of different solving approaches and for advancing the state-of-the-arts of the fields. Traditional competition setting often focuses on producing a ranking between solvers based on their average performance across a wide range of benchmark problems and instances. While this is a sensible way to assess the relative performance of solvers, such ranking does not necessarily reflect the full potential of a solver, especially when we want to utilise a portfolio of solvers instead of a single one for solving a new problem. In this paper, I will describe a portfolio-based analysis method which can give complementary insights into the performance of participating solvers ina competition. The method is demonstrated on the results of the MiniZinc Challenges and new insights gained from the portfolio viewpoint are presented.  \n2012 ACM Subject Classification Theory of computation → Design and analysis of algorithms Keywords and phrases algorithm portfolio, algorithm selection, constraint programming Funding Nguyen Dang: is a Leverhulme Early Career Fellow  \n 1  Introduction  \nIn many algorithmic communities, competitions have been the sources for comparing both established and newly developed solving techniques and for advancing the state-of-the art solving approaches over the years. Examples include the SAT competitions [11, 4] for SAT solvers, the MiniZinc Challenge [19, 20] for constraint solving approaches, the International Planning competitions for the planning community [14, 21] . In a typical competition setting, each participating solver is evaluated on a set of benchmark problems and its average performance across the whole benchmark set is recorded. A ranking among all participants is then produced based on the collected average performance of each solver. While this is a sensible way to assess the relative performance of participating solvers, such ranking does not necessarily reflect the full potential of a solver. For example, a solver with a low overall rank but uniquely performs well on a small subset of the benchmark problems may still be very useful in practice. This is particularly true in the context of algorithm portfoliosand algorithm selection [13, 12], where the aim is construct and utilise a set (a portfolio) of solvers with complementary strengths.  \nThere are recent initiatives [10] that call for a new type of “cooperative” competitions such as the Sparkle SAT challenge 2018 [15] and the Sparkle Planning challenge 2019 [16] . In those challenges, an algorithm selector is built on a portfolio of all participating solvers. Each solveris then ranked based on its marginal contribution [22] to the performance of the selector. More specifically, given a portfolio of n algorithms A = {a1 , a2 , .., an }, the marginal contribution of an algorithm ak to A on an instance set I is defined as P (A, I) − P(A \\ {ak } , I), where P (A, I) is the performance of an algorithm selector constructed from A measured on I. Such type of competitions gives a different view on the performance of a solving approach and can give credits to the solvers that appear to be weak in a traditional competition setting. The algorithm selector will act as the new state-of-art (SOTA) by standing on the shoulders of the participating solvers, and the aim of a newly developed solver would be to join the  \n2 A portfolio-based analysis method for competition result  \nSOTA portfolio and improve the SOTA performance via its marginal contribution to the current portfolio.  \nThe Sparkle setting brings a new and interesting view to competition organisation. However, there are two things we need to consider when applying the setting to a new domain. Firstly, the us","cbCaie90E8WEAUjb","https://ap.wps.com/l/cbCaie90E8WEAUjb","pdf",775214,11,"English","# Introduction\n## A portfolio-based analysis method for competition result","[{\"question\":\"Why is a traditional competition ranking based on average performance insufficient?\",\"answer\":\"It may not reflect a solver’s full potential, since a solver can rank low overall while uniquely performing well on a subset of instances. This matters when leveraging portfolios rather than a single solver.\"},{\"question\":\"How do cooperative competitions like Sparkle change how solver performance is assessed?\",\"answer\":\"They build an algorithm selector from participating solvers’ portfolio and rank solvers by marginal contribution to the selector’s performance on an instance set. This can credit solvers that look weak under official ranking.\"},{\"question\":\"What issue can arise with marginal contribution measurement in portfolio settings?\",\"answer\":\"If two solvers jointly outperform others on a large subset but each is interchangeable, their marginal contributions can be zero, failing to reflect their true impact on overall portfolio performance.\"}]","A portfolio-based analysis method for competition results - read online free | PDF",1791078192,28]