[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84457-en":3,"doc-seo-84457-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84457,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782109480056885918",8,"Research & Report","DEER A Benchmark for Evaluating Deep Research Agents on Expert Report Generation","Recent advances in large language models enable deep research systems to generate expert-level reports via multi-step reasoning and evidence-based synthesis, yet evaluating such outputs remains difficult. Report quality is multidimensional, making criteria selection unclear; LLM-based judges can miss domain-specific errors; and evidence retrieval requires full-report claim verification, not only cited statements. DEER addresses these gaps with an expert-built taxonomy, rubric items, evaluation guidance, and a claim verification architecture for cited and uncited claims plus evidence quality scoring.","DEER: A Benchmark for Evaluating Deep Research Agentson Expert Report Generation  \nJanghoon Han 1 Heegyu Kim 1 Changho Lee 1 Dahm Lee 1 Min Hyung Park 1 Hosung Song 1 Stanley Jungkyu Choi 1 Moontae Lee 1 2 Honglak Lee 1 3  \narXiv :2512 . 17776v 5 [ cs .CL] 13 Jul 2026  \nAbstract  \nRecent advances in large language models have enabled deep research systems that generate expert-level reports through multi-step reasoning and evidence-based synthesis. However, evaluating such reports remains challenging: report quality is multifaceted, making it difficult to determine what to assess and which criteria to use; LLMbased judges may miss errors that require domain expertise to identify; and because deep research relies on retrieved evidence, report-wide claim verification is also necessary. To address these issues, we propose DEER, a benchmark for evaluating expert-level deep research reports. DEER systematizes evaluation criteria with an expertdeveloped taxonomy (7 dimensions, 25 subdimensions) operationalized as 101 fine-grained rubric items. We also provide task-specific Expert Evaluation Guidance to support LLM-based judging. In addition to rubric-based assessment, we propose a claim verification architecture that verifies both cited and uncited claims and quantifies evidence quality. Experiments show that current systems produce structurally plausible, evidence-citing reports, but still struggle to fully satisfy expert-level user requests and achieve logical completeness.  \nBeyond performance comparisons, DEER makes system strengths and limitations interpretable and provides diagnostic signals for improvement.1  \n1. Introduction  \nDriven by rapid advances in large language models (LLMs), automated deep research systems are emerging as a core  \n1LG AI Research 2University of Illinois Chicago 3University of Michigan, Ann Arbor. Correspondence to: Janghoon Han \u003C[hanjh9439@gmail.com](hanjh9439@gmail.com)>.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \n1Code and data: [github.com/hanjanghoon/DEER](github.com/hanjanghoon/DEER).  \nFigure 1. Deep Research System Performance Comparison. Results for five systems on the proposed benchmark.  \ntechnology in both academia and industry (OpenAI, 2025a ; Google, 2025 ; Anthropic, 2025 ; Yang et al., 2025 ; Li et al., 2025b ; Huang et al., 2025 ; Li et al., 2026) . Unlike conventional web search, these systems address complex research queries by decomposing them into multiple steps and dynamically seeking additional information based on intermediate results. Through this process, they integrate information from diverse sources and synthesize multiple perspectives to produce reliable, evidence-based research reports (Xu & Peng, 2025 ; Zhang et al., 2025 ; Java et al., 2026). As a result, deep research systems can achieve strong performance even on challenging benchmark tasks (Mialonet al., 2024 ; Phan et al., 2025) .  \nEarly evaluations of deep research systems relied primarily on complex reasoning benchmarks (Rein et al., 2024 ; Mialon et al., 2024 ; Phan et al., 2025), which indirectly assessed information gathering, hypothesis testing, and multistep reasoning through task performance. Subsequently, deep web search QA benchmarks were introduced to more directly measure systems’ web browsing and information retrieval abilities—core capabilities of deep research—by evaluating multi-step search, information integration, and answer derivation (Wei et al., 2025a ; Krishna et al., 2025 ; Mialon et al., 2024 ; Chen et al., 2025 ; Gou et al., 2026) . More  \nrecently, deep research report benchmarks have emerged to evaluate the quality of generated reports from multiple perspectives, moving beyond simple short-answer-based evaluation (Consult, 2025 ; Coelho et al., 2025 ; Du et al., 2026 ; Wan et al., 2026) .  \nDespite these advancements, existing evaluation methods for deep research systems continue to fac","cbCaicT7DjZj2Gid","https://ap.wps.com/l/cbCaicT7DjZj2Gid","pdf",3097146,1,45,"English","en",105,"# Abstract\n# Introduction\n## Motivation and limitations of existing evaluations\n## DEER overview: taxonomy, rubrics, guidance, and verification","[{\"question\":\"How does DEER verify claims beyond what is explicitly cited in a report?\",\"answer\":\"DEER proposes a claim verification architecture that verifies both cited and uncited claims and quantifies evidence quality, supporting document-wide reliability checks rather than limited citation-marker verification.\"}]",1784195736,113,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"deer-a-benchmark-for-evaluating-deep-research-agents-on-expert-report-generation","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/deer-a-benchmark-for-evaluating-deep-research-agents-on-expert-report-generation/84457/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"How does DEER verify claims beyond what is explicitly cited in a report?","Question",{"text":75,"@type":76},"DEER proposes a claim verification architecture that verifies both cited and uncited claims and quantifies evidence quality, supporting document-wide reliability checks rather than limited citation-marker verification.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":45,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":45,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":98,"slug":130},19,"General","general"]