[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85516-en":3,"doc-seo-85516-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85516,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","JADE Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks","Evaluating agentic AI on open-ended professional tasks faces a core trade-off between rigor and flexibility. Static rubrics ensure reproducibility but cannot cover diverse valid strategies, while LLM-based judging adapts to individual answers yet risks instability and bias. JADE addresses this with a two-layer framework: Layer 1 encodes expert knowledge as reusable evaluation skills for stable criteria, and Layer 2 performs claim-level, report-specific verification with evidence-dependency gating to invalidate refuted claims. Experiments on BizBench show improved stability and uncovered agent failure modes, with transfer to HealthBench and DR.BENCH.","JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional  \nTasks  \nLanbo Lin * 1 Jiayao Liu * 1 Tianyuan Yang * 1 Li Cai 1 2 Yuanwu Xu 1 Lei Wei 1 3 Sicong Xie 1  \nGuannan Zhang 1  \narXiv :2602 .06486v 3 [ cs .AI] 13 Jul 2026  \nAbstract  \nEvaluating agentic AI on open-ended professional tasks faces a fundamental dilemma between rigor and flexibility. Static rubrics provide rigorous, reproducible assessment but fail to accommodate diverse valid response strategies, while LLM-asa-judge approaches adapt to individual responses yet suffer from instability and bias. Human experts address this dilemma by combining domaingrounded principles with dynamic, claim-level assessment. Inspired by this process, we propose JADE, a two-layer evaluation framework. Layer  \n1 encodes expert knowledge as a predefined set of evaluation skills, providing stable evaluation criteria. Layer 2 performs report-specific, claim-level evaluation to flexibly assess diverse reasoning strategies, with evidence-dependency gating to invalidate conclusions built on refuted claims. Experiments on BizBench show that JADE improves evaluation stability and reveals critical agent failure modes missed by holistic LLM-based evaluators. We further demonstrate strong alignment with expert-authored rubrics and effective transfer to HealthBench and DR.BENCH, covering medical and 10-domain professional evaluation settings. Code and data are available at [https:](https:)//[github.com/smiling-world/JADE](github.com/smiling-world/JADE).  \n1. Introduction  \nThe evolution of Large Language Models (LLMs) from chat-bots to reasoning engines has catalyzed the rise of autonomous agentic systems (Naveed et al., 2025 ; Wanget al., 2024a) . By integrating multi-step reasoning with tooluse capabilities, these agents are increasingly deployed in  \n*Equal contribution 1Alibaba International Digital Commerce Group 2Zhejiang University 3Peking University. Correspondence to: Lanbo Lin \u003C[linlanbo.llb@alibaba-inc.com](linlanbo.llb@alibaba-inc.com) > .  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \ncomplex, long-horizon professional workflows (Luo et al., 2025 ; Jimenez et al., 2024) . As capabilities advance, robust evaluation (Yehudai et al., 2025 ; Du et al., 2026) becomes critical for guiding research and ensuring reliability in highstakes applications.  \nMost existing agent benchmarks focus on tasks with verifiable outcomes, such as multi-hop web search (Wei et al., 2025 ; Zhou et al., 2025b ; Chen et al., 2025b), code generation (Jimenez et al., 2024 ; Quan et al., 2025), or toolmediated reasoning (Yao et al., 2025a ; Wang et al., 2026 ; Barres et al., 2025), enabling objective and reproducible evaluation. However, such proxy tasks capture only a limited subset of real-world deployments.  \nIn real-world applications, agents must solve open-ended problems involving strategic analysis, evidence synthesis, and decision-making under evolving conditions (e.g., strategic procurement, market analysis, and professional consulting) . These tasks rarely admit a single “gold” answer, making realistic evaluation an unresolved challenge. For example, a query for “FDA-certified suppliers for stainless steel tumblers with low shipping cost to the USA” admits multiple valid response strategies, whose quality cannot be reliably assessed without consistent professional principles and real-time evidence verification.  \nRecent benchmarks (Du et al., 2026 ; Yao et al., 2025b) explore expert-authored rubrics and LLM-generated checkliststo approximate professional judgment. While promising, these approaches expose a fundamental dilemma: (i) Static checklists ensure stability but lack adaptivity to diverse valid solutions (Arora et al., 2025 ; Ruan et al., 2026 ; Zhu et al., 2025); (ii) LLM-as-a-judge methods (Zheng et al., 2023 ; Du et al., 2026) provide adaptivity through direct scoring or LLM-generate","cbCaioF3LJUuN0Zu","https://ap.wps.com/l/cbCaioF3LJUuN0Zu","pdf",1571821,1,26,"English","en",105,"# Abstract\n# Introduction\n## Stability–Adaptivity Dilemma\n## JADE Two-Layer Evaluation Framework\n## Main Contributions","[{\"question\":\"What problem does JADE address in evaluating agentic AI for open-ended professional tasks?\",\"answer\":\"JADE targets the stability–adaptivity dilemma: static rubrics lack flexibility for multiple valid strategies, while LLM-as-judge approaches can be unstable and biased without expert-grounded, claim-level principles.\"},{\"question\":\"How does JADE achieve both stable evaluation and adaptive flexibility?\",\"answer\":\"JADE uses two layers: Layer 1 encodes expert knowledge as predefined evaluation skills to provide stable criteria, and Layer 2 performs report-specific, claim-level evaluation with evidence-dependency gating to reject conclusions built on refuted claims.\"},{\"question\":\"What benchmarks and domains are used to validate JADE?\",\"answer\":\"Experiments use BizBench, and the method is further demonstrated with alignment to expert-authored rubrics and transfer to HealthBench and DR.BENCH, covering medical and multiple-domain professional evaluation settings.\"}]",1784204118,66,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"jade-expert-grounded-dynamic-evaluation-for-open-ended-professional-tasks","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/jade-expert-grounded-dynamic-evaluation-for-open-ended-professional-tasks/85516/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does JADE address in evaluating agentic AI for open-ended professional tasks?","Question",{"text":75,"@type":76},"JADE targets the stability–adaptivity dilemma: static rubrics lack flexibility for multiple valid strategies, while LLM-as-judge approaches can be unstable and biased without expert-grounded, claim-level principles.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does JADE achieve both stable evaluation and adaptive flexibility?",{"text":80,"@type":76},"JADE uses two layers: Layer 1 encodes expert knowledge as predefined evaluation skills to provide stable criteria, and Layer 2 performs report-specific, claim-level evaluation with evidence-dependency gating to reject conclusions built on refuted claims.",{"name":82,"@type":73,"acceptedAnswer":83},"What benchmarks and domains are used to validate JADE?",{"text":84,"@type":76},"Experiments use BizBench, and the method is further demonstrated with alignment to expert-authored rubrics and transfer to HealthBench and DR.BENCH, covering medical and multiple-domain professional evaluation settings.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]