[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82255-en":3,"doc-seo-82255-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82255,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","Bidirectional Resource Scheduling for Disaggregated and Asynchronous RL Post-Training","Large language model post-training can improve reasoning ability through reinforcement learning, where an iteration alternates rollouts that generate experience and training that updates the policy. Disaggregated asynchronous RL systems raise throughput by separating rollout and training resources, yet idle periods persist under differing hardware, model scales, staleness, hyperparameters, and workload shifts. BiDiRL introduces a hybrid time-space multiplexing design: a hotswitch runtime, a static scheduling-aware planner, and a bidirectional fine-grained scheduler to borrow idle capacity across pools, boosting throughput up to 1.94× without harming convergence.","Bidirectional Resource Scheduling for Disaggregated and Asynchronous RL Post-Training  \nZhiqiang Tan  \n[25b951103@stu.hit.edu.cn](25b951103@stu.hit.edu.cn)[ ](25b951103@stu.hit.edu.cn)School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen Shenzhen, China  \nMaoxin Wang  \n[25s151169@stu.hit.edu.cn](25s151169@stu.hit.edu.cn)[ ](25s151169@stu.hit.edu.cn)School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen Shenzhen, China  \nSijie Wang  \n[25b951105@stu.hit.edu.cn](25b951105@stu.hit.edu.cn)[ ](25b951105@stu.hit.edu.cn)School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen Shenzhen, China  \nYiming Yin  \n[yyin464@connect.hkust-gz.edu.cn](yyin464@connect.hkust-gz.edu.cn)[ ](yyin464@connect.hkust-gz.edu.cn)Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China  \nQiang Wang  \n[qiang.wang@hit.edu.cn](qiang.wang@hit.edu.cn)[ ](qiang.wang@hit.edu.cn)School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen Shenzhen, China  \nXiaowen Chu  \n[xwchu@ust.hk](xwchu@ust.hk)[ ](xwchu@ust.hk)Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China  \narXiv :2607 .09207v1 [ cs .DC] 10 Jul 2026  \nShaohuai Shi∗ [shaohuais@hit.edu.cn](shaohuais@hit.edu.cn)[ ](shaohuais@hit.edu.cn)School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen Shenzhen, China  \nAbstract  \nIt is well established that the reasoning capabilities of large language models (LLMs) can be improved by applying reinforcement learning (RL) in a post-training stage. In a standard RL iteration, the current model (the policy) generates experience through rollouts, and the resulting data is then used to update the policy during training. High-performance RL frameworks such as StreamRL and AReaL employ a disaggregated architecture and asynchronous rollouts to better exploit both rollout and training resources, thereby increasing overall system throughput. Nonetheless, across varying RL setups (e.g., hardware configurations, model scales, staleness levels, and hyperparameters) and under changing workloads, it remains common for both rollout and training resources to experience idle periods.  \nIn this paper, we present BiDiRL, a hybrid time-space multiplexing architecture for asynchronous, disaggregated RL designed to reduce resource idleness. First, we develop a hotswitch runtime that enables rapid switching between rollout and training resources with negligible overhead. Second, we propose a static, scheduling-aware planner based on timeperformance modeling that chooses a hot-switch-friendly resource partition, so that rollout and training durations are roughly balanced at a coarse level. Third, at execution time, we introduce a bidirectional scheduler that further exploits runtime bubbles through fine-grained resource switching, allowing the bottleneck stage to temporarily borrow idle  \n∗ Corresponding author.  \nresources from the other pool. Across a wide range of workloads, datasets, and models on two 32-GPU testbeds, BiDiRL increases RL training throughput by up to 1. 94× compared with state-of-the-art RL systems—including veRL, AReaL, and ROLL—without affecting convergence behavior.  \n1 Introduction  \nReinforcement learning (RL) post-training has become a key technique for improving the reasoning, coding, and tooluse capabilities of large language models (LLMs) [2, 12, 17, 19, 29] . An LLM RL job mainly consists of two stages. In the rollout stage, the actor model takes a batch of promptsand generates responses using an inference engine such as vLLM [9] . We refer to the workers that execute this stage as rollouters. In the training stage, the system scores generated responses, computes policy losses under objectives such as group relative policy optimization (GRPO) [19], and updates the actor model using a training engine such as PyTor","cbCaiafUIAFIaD4u","https://ap.wps.com/l/cbCaiafUIAFIaD4u","pdf",1042199,1,16,"English","en",105,"# Abstract\n# Introduction\n## Post-training RL workflow\n## Colocated vs disaggregated architectures\n## Motivation: idle resource bubbles\n## BiDiRL overview","[{\"question\":\"What problem does BiDiRL address in asynchronous disaggregated RL post-training?\",\"answer\":\"Idle resource bubbles still occur even with asynchronous overlap, caused by fixed disaggregated allocation under varying setups and workloads.\"},{\"question\":\"How does BiDiRL switch between rollout and training resources efficiently?\",\"answer\":\"BiDiRL implements a hotswitch runtime that enables rapid switching between the two resource roles with negligible overhead.\"},{\"question\":\"What scheduling mechanisms does BiDiRL use during execution?\",\"answer\":\"It uses a static scheduling-aware planner to choose a hot-switch-friendly partition, then a bidirectional scheduler at runtime that performs fine-grained switching and lets the bottleneck stage borrow idle resources from the other pool temporarily.\"}]",1784179197,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"bidirectional-resource-scheduling-for-disaggregated-and-asynchronous-rl-post-training","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/bidirectional-resource-scheduling-for-disaggregated-and-asynchronous-rl-post-training/82255/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does BiDiRL address in asynchronous disaggregated RL post-training?","Question",{"text":75,"@type":76},"Idle resource bubbles still occur even with asynchronous overlap, caused by fixed disaggregated allocation under varying setups and workloads.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does BiDiRL switch between rollout and training resources efficiently?",{"text":80,"@type":76},"BiDiRL implements a hotswitch runtime that enables rapid switching between the two resource roles with negligible overhead.",{"name":82,"@type":73,"acceptedAnswer":83},"What scheduling mechanisms does BiDiRL use during execution?",{"text":84,"@type":76},"It uses a static scheduling-aware planner to choose a hot-switch-friendly partition, then a bidirectional scheduler at runtime that performs fine-grained switching and lets the bottleneck stage borrow idle resources from the other pool temporarily.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":28,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]