[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85707-en":3,"doc-seo-85707-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85707,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","EvoCUA-1.5 Online Reinforcement Learning for Multi-turn Computer-Use Agents","Computer-use agents must complete long-horizon goals through repeated interaction with partially observable, multimodal desktop environments. EvoCUA-1.5 moves from offline experience learning to online reinforcement learning, where policies improve from verifiable outcomes in executable sandboxes. The approach tackles multi-turn challenges—context-managed observations, sparse terminal rewards, variable-length trajectories, and slow feedback—via STEPO, policy-aware filtering and pass-rate calibration, Dynamic Tri-Adaptive Curriculum, and an asynchronous RL pipeline. Experiments report 63.2% success on OSWorld-Verified.","arXiv :2607 .09773v 1 [ cs .AI ] 7 Jul 2026  \nEvoCUA-1.5: Online Reinforcement Learning for Multi-turn  \nComputer-Use Agents  \nMianqiu Huang*,†1,2, Taofeng Xue*,†1, Chong Peng*,†1, Jinrui Ding*1 , Sicheng Fan 1,2 , Jiale Hong 1,3 , Yufei Gao 1,4 , Xiaocheng Zhang 1 , Linsen Guo 1 , Xin Yang 1 , Dengchang Zhao 1 , Xiandi Ma 1 , Yuchen Xie 1 , Peng Pei 1 , Xunliang Cai 1 ,  \nXipeng Qiu2  \n1Meituan 2Fudan University 3 Shanghai Jiao Tong University 4Zhejiang University  \nABSTRACT  \nComputer-use agents must solve long-horizon tasks through repeated interaction with partially observable, multimodal desktop environments. Although imitation learning and offline trajectory refinement provide strong priors, static traces cannot cover the causal feedback loop of real computer use: each action changes the screen state, future action space, and recovery options. EvoCUA-1.5 extends self-evolving computer-use agents from offline experience learning to online reinforcement learning, where policies interact with executable sandbox environments and improve from verifiable task outcomes. Online RL in this setting requires more than directly reusing single-turn language-RL recipes. Multi-turn interaction introduces context-managed observations, sparse terminal rewards, variable-length trajectories, and slow environment feedback. EvoCUA-1.5 addresses these challenges with Step-Level Policy Optimization (STEPO), which preserves trajectory-level advantage balance after decomposition into step-level samples; policy-aware filtering and pass-rate calibration over verifiable synthesized tasks; Dynamic Tri-Adaptive Curriculum (DTAC), which combines learnable tasks, difficult positive replay, and controlled infeasible-task exposure; and a fully asynchronous RL infrastructure with staleness control and mini-group batching. Experiments show that these components improve training stability and downstream performance. EvoCUA-1.5 achieves 63.2% success on OSWorld-Verified, outperforming comparable 32B/35B-scale open-weight baselines and even approaching models with significantly larger parameter counts. Overall, EvoCUA-1.5 provides a practical framework for scaling online RL in multi-turn computer-use agents.  \nFigure 1: Performance comparison on OSWorld-Verified. EvoCUA-1.5 achieves 63.2% success rate, outperforming comparable 32B/35B-scale open-weight models.  \n*Equal contribution. †Corresponding authors.  \n1 Introduction  \nDeveloping generalist computer-use agents that can operate graphical user interfaces (GUIs) is a critical step toward autonomous computer use. Unlike specialized automation tools, these agents must perceive complex visual contexts, infer task-relevant states from partial observations, and execute long-horizon workflows across heterogeneous applications. Recent native vision-language models (VLMs) have made substantial progress in unifying perception, reasoning, and action within end-to-end computer-use agents [Bai et al., 2025a, ByteDance Seed Team, 2025, Wang et al., 2025a,b] . Nevertheless, reliable performance in realistic computer-use environments remains difficult, particularly for tasks that require multi-step exploration, recovery from mistakes, and adaptation to dynamic interface states.  \nCurrent progress is still largely driven by imitation learning or offline refinement on fixed interaction traces. Such static data provides useful behavioral priors, but it cannot fully capture the causal feedback loop that characterizes real computer use: each action changes the environment state, which in turn determines the next observation and future action space. Consequently, models trained primarily on offline trajectories are constrained by the coverage of previously collected experience and may struggle with long-tail states, delayed consequences, and failures that emerge only during interaction. This limitation motivates a shift from static trajectory scaling to online experience scaling, in which the policy directly interacts with exe","cbCaikJrrKjzdCu6","https://ap.wps.com/l/cbCaikJrrKjzdCu6","pdf",4600034,1,20,"English","en",105,"# Introduction\n## Motivation and background\n## Limitations of offline approaches\n## EvoCUA-1.5 contributions and design challenges","[{\"question\":\"Why does online reinforcement learning matter for multi-turn computer-use agents?\",\"answer\":\"Offline imitation learning and offline trajectory refinement cannot capture the causal feedback loop of real computer use, where each action changes screen state and future options. Online RL lets the policy interact with executable environments, receive verifiable feedback, and improve from newly generated experience.\"},{\"question\":\"What is the role of Step-Level Policy Optimization (STEPO) in EvoCUA-1.5?\",\"answer\":\"STEPO decomposes trajectory-level advantages into step-level samples while preserving trajectory-level advantage balance. This helps avoid instability caused by naive advantage reuse after decomposition.\"},{\"question\":\"How does EvoCUA-1.5 improve training stability and efficiency in this setting?\",\"answer\":\"It addresses efficiency with policy-aware filtering and pass-rate calibration over verifiable synthesized tasks, and it stabilizes learning using Dynamic Tri-Adaptive Curriculum (DTAC) that mixes learnable tasks, difficult positive replay, and controlled infeasible-task exposure. It also uses a fully asynchronous RL infrastructure with staleness control and mini-group batching.\"}]",1784205726,50,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"evocua-15-online-reinforcement-learning-for-multi-turn-computer-use-agents","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/evocua-15-online-reinforcement-learning-for-multi-turn-computer-use-agents/85707/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why does online reinforcement learning matter for multi-turn computer-use agents?","Question",{"text":74,"@type":75},"Offline imitation learning and offline trajectory refinement cannot capture the causal feedback loop of real computer use, where each action changes screen state and future options. Online RL lets the policy interact with executable environments, receive verifiable feedback, and improve from newly generated experience.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What is the role of Step-Level Policy Optimization (STEPO) in EvoCUA-1.5?",{"text":79,"@type":75},"STEPO decomposes trajectory-level advantages into step-level samples while preserving trajectory-level advantage balance. This helps avoid instability caused by naive advantage reuse after decomposition.",{"name":81,"@type":72,"acceptedAnswer":82},"How does EvoCUA-1.5 improve training stability and efficiency in this setting?",{"text":83,"@type":75},"It addresses efficiency with policy-aware filtering and pass-rate calibration over verifiable synthesized tasks, and it stabilizes learning using Dynamic Tri-Adaptive Curriculum (DTAC) that mixes learnable tasks, difficult positive replay, and controlled infeasible-task exposure. It also uses a fully asynchronous RL infrastructure with staleness control and mini-group batching.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,113,118,121,125,128,132],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":28,"slug":112},6,"Technology","technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":21,"slug":124},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":126,"show_sort_weight":21,"slug":127},"World Cup","world-cup",{"id":129,"doc_module":4,"doc_module_name":45,"category_name":130,"show_sort_weight":129,"slug":131},10,"Lifestyle","lifestyle",{"id":133,"doc_module":4,"doc_module_name":45,"category_name":134,"show_sort_weight":105,"slug":135},19,"General","general"]