[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84175-en":3,"doc-seo-84175-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84175,1374391974468,"Eden","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Measuring Intelligence Beyond Human Scale","How to measure intelligence once human-authored benchmarks saturate is addressed by a new evaluation paradigm. Above human capability, human examiners struggle to craft tasks that are both hard and verifiable, making absolute-scale scoring unreliable. The work proposes relative, model-generated public challenges, whose outcomes are aggregated into an adversarial psychometric rating system. Protocols are provided to reduce private-information attacks, enable judge-sparse adjudication, and scale across verifiable and non-verifiable domains.","arXiv :2607 .07040v 1 [ cs .AI] 8 Jul 2026  \nMeasuring Intelligence Beyond Human Scale  \nJerry Han  \nRafael Moschopoulos Ella Colby  \nKia Ghods Mark Braverman  \nVishrut Goyal Elad Hazan  \nAndrew Tu  \nPrinceton Superalignment∗  \nAbstract  \nHow can we measure intelligence beyond human capability?  \nHuman-authored benchmarks saturate, and above human capability, examiners may not know which tasks are both hard and verifiable. We argue that this difficulty is inherent to absolute-scale evaluation and propose a new paradigm based on relative measurement in which models generate public challenges that separate other systems. Aggregating these outcomes yields an adversarial psychometric rating system that can scale with the systems being measured. We describe practical protocols that reduce incentives for private-information attacks, support judge-free adjudication, and naturally scale with agent capabilities. We instantiate the framework across verifiable and open-ended, non-verifiable domains, illustrating how model-generated evaluation can continue to measure systems beyond the human frontier.  \n1 Introduction  \nHow should intelligence be measured? The question is older than the study of artificial intelligence.  \nThe modern scientific study of intelligence began with the work of Charles Spearman (Spearman, 1904) . Spearman observed that individuals who performed well on one cognitive test tended also to perform well on many others, a phenomenon now known as the positive manifold. He hypothesized that these positive correlations were explained by a single latent factor, which he called general intelligence (g) . This marked a fundamental shift from viewing intelligence as a collection of observable abilities to modeling it as a latent statistical variable inferred from observable behavior, laying the foundations of modern psychometrics. Interestingly, this is also the seed of factor analysis in statistics.  \nLater item-response models made the measurement problem more explicit by jointly modeling subject ability and item difficulty. In the simplest Rasch model (Rasch, 1960), the probability that a subject with ability θ correctly answers an item of difficulty b, and σ is the logistic function, is  \nPr[correct] = σ(θ − b) .  \nThis tradition has two important features. First, intelligence is measured through performance on tasks rather than defined independently of behavior. Second, the tasks are designed by human examiners, which is natural for human populations.  \nMost AI benchmarks inherit this psychometric structure. Systems are evaluated on tasks supplied by humans or environments fixed in advance: game environments, broad academic benchmarks and  \n∗We gratefully acknowledge contributions by Iris Yan and Connor Brown  \nso on (Hendrycks et al. , 2021; Srivastava et al. , 2023; Liang et al. , 2023; Chollet, 2019; White et al. , 2025; Glazer et al. , 2024; Phan et al. , 2026; Kiela et al. , 2021) . This becomes a scaling problem atthe frontier: benchmarks saturate, and new ones require humans to generate tasks that are difficult and verifiable. Above human expert level, the bottleneck is not only the number of questions, but the examiner’s ability to formulate discriminating questions.  \nA different approach to measuring intelligence. A different starting point is relative measurement. Turing’s imitation game measures machine intelligence by comparison with human behavior before a human judge, rather than by an absolute score on a fixed test (Turing, 1950) . Modern pairwise preference and arena methods similarly compare systems through judged interactions (Zheng et al. , 2023; Chiang et al. , 2024) . These protocols point toward relative evaluation, but the tasks or judgments are still largely human-supplied. A notable exception is MathDuels (Xu et al. , 2026), which explores model-generated mathematical pairwise challenges as an evaluation primitive.  \nHowever, purely pairwise comparative protocols are vulnerable to failure modes","cbCaimaakI5iQJDV","https://ap.wps.com/l/cbCaimaakI5iQJDV","pdf",399777,1,26,"English","en",105,"# Introduction\n## Relation to existing work","[{\"question\":\"Why do human-authored intelligence benchmarks fail near or beyond human capability?\",\"answer\":\"They saturate, and examiners above human expert level face a bottleneck in formulating discriminating tasks that are both hard and verifiable.\"},{\"question\":\"What is the core idea of the proposed evaluation paradigm?\",\"answer\":\"Replace pairwise comparison with separative measurement, where a challenge is judged by the variance it induces across a population of solvers.\"},{\"question\":\"How does adversarial psychometrics improve scalability and adjudication?\",\"answer\":\"Model-generated challenges yield a richer statistical signal that can scale beyond the human frontier and support sparse or even judge-free adjudication while reducing incentives for private-information attacks.\"}]",1784193638,66,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"measuring-intelligence-beyond-human-scale","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/measuring-intelligence-beyond-human-scale/84175/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do human-authored intelligence benchmarks fail near or beyond human capability?","Question",{"text":75,"@type":76},"They saturate, and examiners above human expert level face a bottleneck in formulating discriminating tasks that are both hard and verifiable.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the core idea of the proposed evaluation paradigm?",{"text":80,"@type":76},"Replace pairwise comparison with separative measurement, where a challenge is judged by the variance it induces across a population of solvers.",{"name":82,"@type":73,"acceptedAnswer":83},"How does adversarial psychometrics improve scalability and adjudication?",{"text":84,"@type":76},"Model-generated challenges yield a richer statistical signal that can scale beyond the human frontier and support sparse or even judge-free adjudication while reducing incentives for private-information attacks.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]