[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85496-en":3,"doc-seo-85496-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85496,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","DR-Arena: an Automated Evaluation Framework for Deep Research Agents","Large Language Models increasingly act as Deep Research (DR) agents for autonomous investigation and information synthesis, making reliable evaluation a key bottleneck. Existing benchmarks rely on static datasets, limiting task generality, becoming temporally misaligned, and risking data contamination. DR-Arena introduces a fully automated evaluation framework using real-time Information Trees from fresh web trends and an automated Examiner that generates structured tasks for deep reasoning and wide coverage. An adaptive evolvement loop escalates complexity based on live performance.","DR-Arena: an Automated Evaluation Framework for Deep Research Agents  \nYiwen Gao1 * Ruochen Zhao2 * Yang Deng3 Wenxuan Zhang4†  \n1National University of Singapore 2Nanyang Technological University  \n3 Singapore Management University 4 Singapore University of Technology and Design  \n[yiwen_gao@u.nus.edu](yiwen_gao@u.nus.edu)[ ](yiwen_gao@u.nus.edu)[ydeng@smu.edu.sg](ydeng@smu.edu.sg)  \n[ruochen002@e.ntu.edu.sg](ruochen002@e.ntu.edu.sg)[ ](ruochen002@e.ntu.edu.sg)[wxzhang@sutd.edu.sg](wxzhang@sutd.edu.sg)  \n[https://github.com/iNLP-Lab/DR-Arena](https://github.com/iNLP-Lab/DR-Arena)  \narXiv :2601 . 10504v 3 [ cs .CL] 12 Jul 2026  \nAbstract  \nAs Large Language Models (LLMs) increasingly operate as Deep Research (DR) Agents capable of autonomous investigation and information synthesis, reliable evaluation of their task performance has become a critical bottleneck. Current benchmarks predominantly rely on static datasets, which suffer from several limitations: limited task generality, temporal misalignment, and data contamination. To address these, we introduce DR-Arena, a fully automated evaluation framework that pushes DR agents to their capability limits through dynamic investigation. DR-Arena constructs real-time Information Trees from fresh web trends to ensure the evaluation rubric is synchronized with the live world state, and employs an automated Examiner to generate structured tasks testing two orthogonal capabilities:  \nDeep reasoning and Wide coverage. DR-Arena further adopts Adaptive Evolvement Loop, a state-machine controller that dynamically escalates task complexity based on real-time performance, demanding deeper deduction or wider aggregation until a decisive capability boundary emerges. Experiments with six advanced DR agents demonstrate that DR-Arena achievesa Spearman correlation of 0.94 with the LMSYS Search Arena leaderboard. This represents state-of-the-art alignment with human preferences without any manual efforts, validating DR-Arena as a reliable alternative for costly human adjudication.  \n1 Introduction  \nDeep Research (DR) agents, such as OpenAI Deep Research (OpenAI, 2025) and Perplexity Deep Research (Perplexity AI, 2025), have rapidly gained adoption and are now widely used for complex information-seeking tasks. Unlike traditional search engines where users need to browse multiple websites manually, DR agents act as autonomous  \n* Equal contribution.  \n† Corresponding author.  \nresearch agents that conduct multi-step investigations over extended horizons, iteratively retrieving, cross-referencing, and synthesizing evidence from the live web to produce structured and citationbacked reports (Nakano et al., 2022 ; Shao et al., 2024 ; Qin et al., 2023) . As DR agents are increasingly deployed in real-world and high-stakes analytical settings, efficient and reliable evaluation of their capabilities has become a pressing challenge.  \nRecent efforts have begun to evaluate deep research capabilities by constructing dedicated datasets for multi-step web-based investigation (Mialon et al., 2024 ; Wong et al., 2025 ; Lan et al., 2025) . While these benchmarks provide useful performance indicators, they exhibit three fundamental limitations: 1) limited task generality, as taskcentric and aspect-specific dataset construction restricts evaluation to predefined investigation patterns and weakens transferability to real-world research settings; 2) temporal misalignment, since static benchmarks inevitably decay as underlying facts evolve, resulting in evaluation against outdated ground truth; and 3) data contamination, whereby persistent and widely reused datasets increasingly appear in model training corpora, resulting in parametric memorization rather than genuine reasoning and evidence synthesis (Ravaut et al., 2025) . Collectively, these issues reflect a structural mismatch with DR agents, which are designed to operate in open evolving environments that require adaptive exploration and reasoning.  \nTo address th","cbCaiiWafP2OuIUD","https://ap.wps.com/l/cbCaiiWafP2OuIUD","pdf",1729343,1,23,"English","en",105,"# Introduction\n## Deep Research agents and evaluation challenges\n## DR-Arena framework overview\n## Adaptive Evolvement and capability stress tests","[{\"question\":\"Why do current DR evaluation benchmarks struggle to assess real-world deep research agents?\",\"answer\":\"They often use static datasets, which reduces task generality, causes temporal misalignment as facts change, and can lead to data contamination through reuse in training corpora.\"},{\"question\":\"How does DR-Arena ensure its evaluation remains synchronized with the live world state?\",\"answer\":\"DR-Arena builds real-time Information Trees by scraping informative websites from current web trends, so the evaluation rubric matches the latest information landscape.\"},{\"question\":\"What capabilities does DR-Arena test and how does it adapt difficulty during evaluation?\",\"answer\":\"It stresses two dimensions: multi-hop reasoning (Depth) and information gathering (Width). An Adaptive Evolvement Loop escalates task complexity based on real-time performance and uses targeted follow-up rounds to probe specific failure types.\"}]",1784204013,58,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"dr-arena-an-automated-evaluation-framework-for-deep-research-agents","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/dr-arena-an-automated-evaluation-framework-for-deep-research-agents/85496/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do current DR evaluation benchmarks struggle to assess real-world deep research agents?","Question",{"text":75,"@type":76},"They often use static datasets, which reduces task generality, causes temporal misalignment as facts change, and can lead to data contamination through reuse in training corpora.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does DR-Arena ensure its evaluation remains synchronized with the live world state?",{"text":80,"@type":76},"DR-Arena builds real-time Information Trees by scraping informative websites from current web trends, so the evaluation rubric matches the latest information landscape.",{"name":82,"@type":73,"acceptedAnswer":83},"What capabilities does DR-Arena test and how does it adapt difficulty during evaluation?",{"text":84,"@type":76},"It stresses two dimensions: multi-hop reasoning (Depth) and information gathering (Width). An Adaptive Evolvement Loop escalates task complexity based on real-time performance and uses targeted follow-up rounds to probe specific failure types.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]