[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84406-en":3,"doc-seo-84406-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84406,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks","Large language models and multimodal large language models are enabling proactive agents that operate everyday tools in real environments. Existing benchmarks often rely on sandboxed setups and single-turn evaluation, and their scenario-based taxonomies mix different capabilities, obscuring failure root causes. UniClawBench introduces a capability-driven framework with five core capabilities, 400 bilingual real-world tasks, and live Docker-based step checkpoints plus a closed-loop executor–supervisor–user evaluation strategy. Experiments compare models and agent frameworks.","UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks  \narXiv :2607 .08768v 1 [ cs .CL] 9 Jul 2026  \nZhekai Chen1∗ Chengqi Duan1 ∗ Kaiyue Sun1∗ Bohao Li1 Yuqing Wang1  \nManyuan Zhang2† Xihui Liu1†  \n1HKU MMLab 2Meituan  \nFigure 1: Overview of UniClawBench. UniClawBench consists of 400 bilingual real-world tasks spanning 5 core capabilities: multimodal, long-context, skill usage, exploration, and cross-platform. We propose a three-role closed-loop evaluation framework, where an executor agent performs tasks in real environments, a supervisor evaluates trajectories and artifacts using hidden rubrics, and a user simulator provides natural feedback based on executor’s performance and supervisor’s signal, enabling multi-turn interaction. We conduct two sets of experiments to evaluate cross-model and cross-framework performance. The capability-level results reveal that framework choice exerts a stronger influence than model choice.  \nAbstract  \nThe rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operating everyday tools and assisting users in real-world environments. However, existing benchmarks struggle to evaluate such agents effectively, as they often rely on sandboxed environments and single-turn evaluation paradigms. Moreover, their scenario-based task taxonomies mix multiple model capabilities within the same  \ntask category, making it difficult to identify the root causes of agent failures. To address these limitations, we introduce UniClawBench, the first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings. UniClawBench is built around five foundational model capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross  \nPlatform Coordination. Based on these capabilities, we design 400 bilingual realworld tasks. Unlike previous benchmarks that rely on static, pre-recorded answers, our benchmark evaluates agents in live Docker containers using fine-grained, step by-step completion checkpoints. Furthermore, we design a closed-loop evaluation  \n∗ Equal contribution, listed alphabetically †Corresponding Author  \nPreprint.  \nstrategy comprising an executor agent, a hidden supervisor agent, and a user agent to simulate realistic multi-turn human feedback without leaking grading criteria. To disentangle base model capabilities from framework-level design choices, we evaluate state-of-the-art models under multiple agent frameworks.  \nThrough comprehensive comparisons across both models and frameworks, we show how base model capabilities and agent framework designs jointly shape performance in real-world environments. To facilitate future research, we make our benchmark and code publicly available at [https://github.com/HKU-MMLab/](https://github.com/HKU-MMLab/)[ ](https://github.com/HKU-MMLab/)UniClawBench.  \n1 Introduction  \nLarge language models and multimodal large language models have evolved from text-centric assistants into autonomous agents capable of executing complex, multi-step tasks in real-world environments [48, 36] . This evolution has given rise to proactive agents, which assist user by directly control everyday tools like browsers and terminals. Representative platforms such as OpenClaw [39] and Nanobot [35] have moved beyond isolated task execution toward 24-7 personal assistance, effectively serving as digital coworkers that handle the full spectrum of daily work. These agents are tasked with navigating complex, real-world scenarios that span multiple modalities and platforms. This rapid evolution raises a pressing question: How can we systematically measure their true capabilities in real-world settings?  \nSignificant efforts have been devoted to building agent benchmarks [20, 26] . However, we identify three structural limitations that prevent them from adequately evaluating modern proactive agents. First, existing ","cbCaiobctVbsQSKQ","https://ap.wps.com/l/cbCaiobctVbsQSKQ","pdf",2044867,1,33,"English","en",105,"# Introduction\n## Motivation and limitations of existing benchmarks\n## Challenges in real-world, closed-loop evaluation\n# UniClawBench overview\n## Capability-driven benchmark design\n## Closed-loop three-role evaluation framework\n# Experiments\n## Cross-model evaluation\n## Cross-framework evaluation\n# Results and findings\n## Capability-level impact of framework choice","[{\"question\":\"Why do existing proactive-agent benchmarks fail to evaluate real-world capability accurately?\",\"answer\":\"They often use sandboxed environments and single-turn evaluation, and their scenario-based task categories mix multiple abilities. This makes it hard to identify the specific source of agent failures.\"},{\"question\":\"What core capabilities does UniClawBench evaluate?\",\"answer\":\"UniClawBench evaluates five capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross-Platform Coordination.\"},{\"question\":\"How does UniClawBench prevent leaking evaluation criteria while still simulating user feedback?\",\"answer\":\"It uses a closed-loop setup with an executor agent, a hidden supervisor that evaluates using hidden rubrics, and a user simulator that provides natural feedback based on executor performance and supervisor signals, without exposing grading criteria.\"}]",1784195367,83,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"uniclawbench-a-universal-benchmark-for-proactive-agents-on-real-world-tasks","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/uniclawbench-a-universal-benchmark-for-proactive-agents-on-real-world-tasks/84406/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why do existing proactive-agent benchmarks fail to evaluate real-world capability accurately?","Question",{"text":74,"@type":75},"They often use sandboxed environments and single-turn evaluation, and their scenario-based task categories mix multiple abilities. This makes it hard to identify the specific source of agent failures.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What core capabilities does UniClawBench evaluate?",{"text":79,"@type":75},"UniClawBench evaluates five capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross-Platform Coordination.",{"name":81,"@type":72,"acceptedAnswer":82},"How does UniClawBench prevent leaking evaluation criteria while still simulating user feedback?",{"text":83,"@type":75},"It uses a closed-loop setup with an executor agent, a hidden supervisor that evaluates using hidden rubrics, and a user simulator that provides natural feedback based on executor performance and supervisor signals, without exposing grading criteria.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]