[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84970-en":3,"doc-seo-84970-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84970,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning","Recent reinforcement learning advances show strong potential for agentic large language model (LLM) tasks, yet most work targets single-task setups while deployments require one generalist policy handling multiple tasks simultaneously. This study identifies exploration-exploitation pace mismatch across tasks: easier tasks can converge early to low-entropy policies that block learning on harder tasks, while harder tasks can push easier ones back toward exploration. Entropy pacing policy optimization coordinates entropy across tasks via task-wise dynamic clipping, improving multi-task optimization stability over prior methods.","arXiv :2607 .07 178v 1 [ cs .LG] 8 Jul 2026  \nEntropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning  \nZetian Hu1* , Shunyu Liu1 , Junjie Zhang1 , Yongcheng Jing1 , Ting-En Lin2 , Yongbin Li2 , Dacheng Tao1†  \n1 Generative AI Lab, College of Computing and Data Science,  \nNanyang Technological University, Singapore 639798  \n2Tongyi Lab, Alibaba Group  \nAbstract  \nRecent breakthroughs of Reinforcement Learning (RL) have highlighted its potential for complex agentic Large Language Model (LLM) tasks. However, existing efforts largely focus on single-task settings, whereas real-world deployment necessitates a generalist agent capable of solving multiple tasks simultaneously. In this work, we identify a critical yet underexplored phenomenon in multi-task agentic RL: different tasks can exhibit exploration-exploitation pace mismatch. Specifically, easier tasks may converge early to low-entropy policies that hinder learning on harder tasks, while harder tasks can, in turn, push easier tasks back toward high-entropy exploration. This back-and-forth interaction creates inter-task entropy crossovers and frequent entropy spikes. Inspired by this observation, we introduce Entropy Pacing Policy Optimization (EPPO) for multi-task agentic LLMs, which coordinates entropy across tasks to stabilize multi-task optimization. At the core of EPPO is a task-wise dynamic clipping mechanism that replaces the fixed clipping threshold in Group Relative Policy Optimization (GRPO) with a task entropyaware adaptive bound, tightening updates for over-confident tasks while relaxing them for under-explored ones. Experiments on the multi-task agentic benchmarks demonstrate that the proposed EPPO yields results superior to its counterparts.  \n1 Introduction  \nRecent advances in large language models (LLMs) have moved post-training beyond high-quality single responses toward interactive agents that act over long horizons and use tools to accomplish goals in diverse environments [Plaat et al., 2025, Wang et al., 2025, Luo et al., 2025] . Compared with single-turn tasks such as math problem solving [Guo et al., 2025] or code generation [Guo et al., 2024], agentic settings involve sparse, delayed feedback and thus naturally call for Reinforcement Learning (RL), where a policy must plan, explore, recover from errors, and adapt over multiple steps [Zhang et al., 2025a, 2026] . RL-trained LLM agents have demonstrated strong performance across diverse tasks, including search and research [Feng et al., 2025, Jin et al., 2025], code generation [Zhang et al., 2024], and GUI navigation [Liu et al., 2025b] . However, most existing work optimizes agents for a single task or domain [Wu et al., 2025, Li et al., 2025b], which limits transfer and complicates real-world deployment. This motivates multi-task agentic RL: training one unified policy that can generalize across heterogeneous agentic tasks.  \nScaling multi-task agentic RL remains difficult. Tasks of different difficulty are highly heterogeneous, and agentic environments vary widely in horizon length, feedback density, and tool-use structure,  \n* Email: [zetian.hu@ntu.edu.sg](zetian.hu@ntu.edu.sg)  \n†Corresponding authors: [dacheng.tao@gmail.com](dacheng.tao@gmail.com)  \nPreprint.  \noften causing imbalance and negative transfer in a shared policy [Zhang et al., 2025b] . Existing multi-task approaches roughly fall into two families. (i) Expert-training and distillation methods such as MiMo-V2-Flash [Xiao et al., 2026] and DeepSeek-V3.2 [Liu et al., 2025a] first train domain specialists, then distill or integrate their expertise into a unified model, which alleviates capability imbalance and improves knowledge integration across domains. (ii) End-to-end joint training methods train a single policy directly with multi-task balancing and scalable environments [Zhang et al., 2025c, Li et al., 2025a, Dai et al., 2024] . Among them, AgentRL [Zhang et al., 2025b] makes an important step for multi-task a","cbCainRf8DUTjCbi","https://ap.wps.com/l/cbCainRf8DUTjCbi","pdf",504981,1,18,"English","en",105,"# Introduction\n## Multi-task agentic RL motivation\n## Exploration-exploitation pace mismatch and entropy dynamics\n## Proposed EPPO approach","[{\"question\":\"What problem does the paper identify in multi-task agentic reinforcement learning?\",\"answer\":\"It identifies an exploration-exploitation pace mismatch across tasks, where easy tasks can collapse to low-entropy policies that hinder harder tasks, while harder tasks can destabilize already-converged tasks.\"},{\"question\":\"How does EPPO differ from GRPO in handling policy updates?\",\"answer\":\"EPPO replaces GRPO’s global fixed clipping range with a task-wise dynamic clipping mechanism that uses an entropy-aware adaptive bound.\"},{\"question\":\"Why does the paper use task entropy as a key signal?\",\"answer\":\"The method treats policy entropy as a proxy for each task’s exploration-exploitation state, guiding how tightly or loosely updates are clipped to stabilize joint training.\"}]",1784199846,45,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"entropy-pacing-policy-optimization-for-multi-task-agentic-reinforcement-learning","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/entropy-pacing-policy-optimization-for-multi-task-agentic-reinforcement-learning/84970/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the paper identify in multi-task agentic reinforcement learning?","Question",{"text":74,"@type":75},"It identifies an exploration-exploitation pace mismatch across tasks, where easy tasks can collapse to low-entropy policies that hinder harder tasks, while harder tasks can destabilize already-converged tasks.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does EPPO differ from GRPO in handling policy updates?",{"text":79,"@type":75},"EPPO replaces GRPO’s global fixed clipping range with a task-wise dynamic clipping mechanism that uses an entropy-aware adaptive bound.",{"name":81,"@type":72,"acceptedAnswer":82},"Why does the paper use task entropy as a key signal?",{"text":83,"@type":75},"The method treats policy entropy as a proxy for each task’s exploration-exploitation state, guiding how tightly or loosely updates are clipped to stabilize joint training.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]