[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83798-en":3,"doc-seo-83798-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83798,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","HAS-Bench: Evaluating LLM-Based Human-Agent Systems under Configurable Human Participation","Large language models increasingly collaborate with humans as active co-workers rather than passive task requesters. HAS-FRAMEWORK models humans and LLM-powered agents as first-class participants in an interaction graph with explicit roles, permissions, communication paths, and action authority. Building on this, HAS-BENCH evaluates human-agent systems with configurable human participation across agency levels, interaction channels, and persona policies. The benchmark measures both task outcomes and collaboration behaviors such as clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost, across six domains.","arXiv :2607 .04329v 1 [ cs .AI ] 5 Jul 2026  \nHAS-BENCH: EVALUATING LLM-BASED HUMANAGENT SYSTEMS UNDER CONFIGURABLE HUMAN PARTICIPATION  \nYaozu Wu1 ∗ Wei-Chieh Huang2 ∗ Jizhou Guo2 ∗ Dongyuan Li1 Renhe Jiang1 Henry Peng Zou2  \nChunyu Miao2 Shanghao Li2 Weizhi Zhang2 WeiWei Ye1 Yankai Chen3 4 Meng Zhang5  \nXue Liu3 4 Philip S. Yu2  \n1 The University of Tokyo 2University of Illinois Chicago 3MBZUAI  \n4McGill University 5 Zhejiang University [yaozuwu279@gmail.com](yaozuwu279@gmail.com) , [whuang80@uic.edu](whuang80@uic.edu) , [sjtu18640985163@sjtu.edu.cn](sjtu18640985163@sjtu.edu.cn) ,  \n[lidy@csis.u-tokyo.ac.jp](lidy@csis.u-tokyo.ac.jp) , [jiangrh@csis.u-tokyo.ac.jp](jiangrh@csis.u-tokyo.ac.jp)  \nABSTRACT  \nLarge language models increasingly operate in settings where humans are ac  \ntive collaborators rather than passive task providers. We introduce HAS  \nFRAMEWORK, a graph-based framework that represents humans and LLM  \npowered agents as first-class participants with explicit roles, permissions, commu  \nnication paths, and action authority. Building on this framework, HAS-BENCH  \nevaluates Human-Agent Systems under configurable human participation across  \nagency levels, interaction channels, and persona policies. The benchmark mea  \nsures both task outcomes and process-level collaboration behavior, including clar  \nification quality, feedback utilization, control calibration, safety, initiative, and in  \nteraction cost. Experiments across six domains show that human participation can  \nsubstantially improve task completion and failure recovery, but the gains depend  \non when, how, and by whom human input is exercised.  \n1 INTRODUCTION  \nRecent advances in Large Language Models (LLMs) have enabled a new generation of HumanAgent Systems (HAS), which integrate the strengths of both humans and LLM-based agents to collaboratively solve complex tasks (Zou et al., 2025) . Unlike conventional agentic systems that often treat users as passive information providers (Wang et al., 2024a; Durante et al., 2024), HAS support dynamic, bidirectional collaboration, where the human can shape the agent’s actions and decisions through clarification, feedback, and control, exercised both at the agent’s request and on the human’s own initiative (Shao et al., 2024; Miao et al., 2025; Barres et al., 2025) . Such systems are increasingly explored in real-world scenarios, where human expertise, contextual judgment, or real-time intervention is critical, including travel planning (Qian et al., 2025), financial decisionmaking (Xu et al., 2025), and autonomous driving (Wu et al., 2025) .  \nDespite this growing interest, existing benchmarks only partially capture the full interaction dynamics of HAS. General agent benchmarks (Liu et al., 2024; Zhou et al., 2024a; Xie et al., 2024; Trivedi et al., 2024) evaluate autonomous agents in realistic interactive environments, but typically treat users as static task providers rather than active collaborators. User-agent interaction benchmarks (Wang et al., 2024b; Lu et al., 2025; Qian et al., 2025; Barres et al., 2025) introduce multiturn dialogue, tool use, preference elicitation, or dual-control settings, yet they largely focus on single agent-user scenarios with fixed user roles, and do not systematically examine how varying of human participation affect coordination, safety, and task completion. In contrast, multi-agent benchmarks (Zhu et al., 2025; Zhou et al., 2024b; Sun et al., 2025; Emde et al., 2026) study coordination and competition among LLM agents, but rarely model humans as first-class participants with explicit agency, permissions, responsibilities, and intervention channels. As a result, there  \n*Equal contribution.  \n†Corresponding author.  \nHAS-Bench  \nA benchmark for LLM agents under configurable human participation  \nHAS-BENCH CONFIGURES TWO AXES  \n3 INTERACTION CHANNELS  \n5 HUMAN AGENCY LEVELS automation  \nDEFAULT  \nboth contribute augmentation  \nSAME TASK · THREE RUNS airline_A_ 002—rebook M05","cbCairUEdCkzpFMt","https://ap.wps.com/l/cbCairUEdCkzpFMt","pdf",2059859,1,52,"English","en",105,"# Introduction\n## Motivation and research gap\n## HAS-Framework and HAS-Bench overview\n# Evaluation principles and benchmark design\n## First-class human participation model\n## Interaction channels and metrics","[{\"question\":\"What problem does HAS-BENCH target in evaluating LLM-based human-agent systems?\",\"answer\":\"It addresses the lack of benchmarks that fully capture interaction dynamics when humans are active collaborators rather than static task providers, including how varying human participation affects coordination, safety, and task completion.\"},{\"question\":\"How does the HAS-FRAMEWORK represent humans and LLM agents?\",\"answer\":\"It uses a graph-based execution framework where humans and LLM-powered agents are first-class participants with explicit roles, permissions, communication paths, and action authority.\"},{\"question\":\"Which aspects does HAS-BENCH measure beyond final task results?\",\"answer\":\"It measures process-level collaboration behaviors such as clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost.\"}]",1784190478,131,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"has-bench-evaluating-llm-based-human-agent-systems-under-configurable-human-participation","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/has-bench-evaluating-llm-based-human-agent-systems-under-configurable-human-participation/83798/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does HAS-BENCH target in evaluating LLM-based human-agent systems?","Question",{"text":75,"@type":76},"It addresses the lack of benchmarks that fully capture interaction dynamics when humans are active collaborators rather than static task providers, including how varying human participation affects coordination, safety, and task completion.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the HAS-FRAMEWORK represent humans and LLM agents?",{"text":80,"@type":76},"It uses a graph-based execution framework where humans and LLM-powered agents are first-class participants with explicit roles, permissions, communication paths, and action authority.",{"name":82,"@type":73,"acceptedAnswer":83},"Which aspects does HAS-BENCH measure beyond final task results?",{"text":84,"@type":76},"It measures process-level collaboration behaviors such as clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]