[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-273395-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-273395-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","the-metanym-game-an-llm-benchmark-without-ground-truth-that-rises-with-the-models-it-measures","The Metanym Game - An LLM Benchmark Without Ground Truth That Rises With the Models It Measures","","The Metanym Game proposes a ground-truth-free LLM benchmark built around analogy as the core of language intelligence. Models collaboratively generate templated sets of analogous factual statements, then grade one another using only internal ratings across multiple subjective and objective criteria. Ground truth is replaced by a spectral decomposition of the factual-rating matrix, jointly identifying generator and judge roles. Contamination resistance is achieved by fresh item generation, and benchmark scale adapts via an official council-of-peers mechanism.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/the-metanym-game-an-llm-benchmark-without-ground-truth-that-rises-with-the-models-it-measures/273395/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/the-metanym-game-an-llm-benchmark-without-ground-truth-that-rises-with-the-models-it-measures/273395.png","ImageObject",300,407,{"name":42,"@type":43},"Anda","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-16","2026-09-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",12,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"How does the metanym game benchmark avoid using predetermined ground truth?","Question",{"text":63,"@type":64},"The benchmark uses game rules only: players generate the items in play and grade them with no answer key or external labels. Every score comes from players’ own ratings.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What is used as a substitute for ground truth in the benchmark’s scoring?",{"text":68,"@type":64},"Ground truth is replaced by an SVD-based spectral method on the factual rating matrix, which judges players as both generators and judges.",{"name":70,"@type":61,"acceptedAnswer":71},"Why is the benchmark resistant to data contamination or leakage?",{"text":72,"@type":64},"Items are produced fresh during each run, so there is no fixed test set that a later model could have seen during training.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},273395,1789471239,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},962075006959,"https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297","arXiv : 2606 .2 1008v 3 [cs .CL] 20 Aug 2026  \nThe Metanym Game: An LLM Benchmark Without Ground Truth That Rises With the Models It  \nMeasures  \nDavid Nordfors ([david.nordfors@archetypes.ai](david.nordfors@archetypes.ai) )  \nThe Metanym Game: An LLM Benchmark Without Ground Truth That Rises With the Models It Measures  \nAbstract  \nWe present evidence that analogy is at the core of LLM intelligence. In our benchmark, LLMs compete in generating sets of analogous statements and rate each other’s sets on their own understandings of factual correctness, beauty, intelligence, distinctness, length, and structural diversity. Nothing enters from outside: the only given is the game rules; every item is generated in play; the scores come from the players’ ratings alone. Ground truth is replaced by the SVD of the factual rating matrix, which scores players as generators and judges at once — to our knowledge the first eigen-equation that judges the judges for an LLM council-of-peers. For subjective criteria like beauty, judges are weighted by their rating consistency. The best generators turn out to be middling judges. GPQA Diamond — difficult multiple-choice questions written by human experts—could not be more different in method, yet the two benchmarks correlate at Pearson 􀔡 = 0.97, 95% CI [0.92, 0.99]; no leakage could be found. A council of the five best issues the official ratings; its contestable seats let the benchmark scale to any number of players and rise with the models it measures—a candidate steering signal for self-improving AI. Playing interweaves at least eight constructs of intelligence; the total scores the broad composite, the components allow reductionistic analysis. Every number recomputes from a released package on GitHub.  \n1 . Introduction  \nNearly every benchmark for machine intelligence needs a predetermined ground truth — golden keys and labels, oracle models, human panels. The benchmark reported here needs none of that. It is a game where frontier language models compete in making up analogies and then grade one another, and that grading is the single source of every score: no human raters, no answer key, nothing to look up.  \nThe test is the metanym game. A player authors, from nothing, a context template—a paragraph of fixed wording with open slots—together with the metanym sets that fill it, each set instantiating the template as a factually true description of a different  \ndomain. Consider this passage describing cell signalling, from an old version of the Wikipedia article as it was worded when we first conceptualised the idea:  \nCELL SIGNALING is part of a complex system of communication that governs basic CELLULAR activities and coordinates CELL actions. The ability of CELLS to perceive and correctly respond to their MICROENVIRONMENT is the basis of development, TISSUE repair, and IMMUNITY as well as normal TISSUE HOMEOSTASIS. Errors in CELLULAR information processing are responsible for DISEASES. By understanding CELL SIGNALING, DISEASES may be treated effectively. SYSTEMS BIOLOGY research helps us to understand the underlying structure of CELL SIGNALING networks and how changes in these networks may affect the transmission and flow of information. CELL SIGNALING [is mostly thought of as] signaling between CELLS of a single ORGANISM. However, CELL SIGNALING may also occur between the CELLS of two different ORGANISMS. (adapted from Wikipedia’s article on cell signalling)  \nNow substitute the set of marked keywords with another set:  \nHUMAN LANGUAGE is part of a complex system of communication that governs basic HUMAN activities and coordinates HUMAN actions. The ability of HUMANS to perceive and correctly respond to their ENVIRONMENT is the basis of development, COMMUNITY repair, and RESILIENCE as well as normal COMMUNITY EQUILIBRIUM. Errors in HUMAN information processing are responsible for DYSFUNCTIONS. By understanding HUMAN LANGUAGE, DYSFUNCTIONS may be treated effectively. SOCIOLOGY research help","cbCaitjnaqnpM5d3","https://ap.wps.com/l/cbCaitjnaqnpM5d3","pdf",1552549,69,"English","# 1. Introduction\n## The test is the metanym game\n## Metanyms and templated analogous statements\n## Ground-truth-free scoring and contamination resistance\n## Matrix-based judge competence and council grading\n# 2. Canonical findings","[{\"question\":\"How does the metanym game benchmark avoid using predetermined ground truth?\",\"answer\":\"The benchmark uses game rules only: players generate the items in play and grade them with no answer key or external labels. Every score comes from players’ own ratings.\"},{\"question\":\"What is used as a substitute for ground truth in the benchmark’s scoring?\",\"answer\":\"Ground truth is replaced by an SVD-based spectral method on the factual rating matrix, which judges players as both generators and judges.\"},{\"question\":\"Why is the benchmark resistant to data contamination or leakage?\",\"answer\":\"Items are produced fresh during each run, so there is no fixed test set that a later model could have seen during training.\"}]","The Metanym Game - An LLM Benchmark Without Ground Truth That Rises With the Models It Measures | PDF",174]