[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83942-en":3,"doc-seo-83942-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83942,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents","Long-horizon agentic LLMs face a practical bottleneck: extended interaction trajectories can exceed the model’s finite context window before a task is completed. Context compaction summarizes earlier interaction states and resumes rollout from a compressed representation, yet integrating compaction into reinforcement learning has remained underexplored. CompactionRL trains long-horizon agentic LLMs to jointly optimize task execution and summary generation, using token-level loss normalization and cross-trajectory generalized advantage estimation. Experiments on agentic coding show consistent Pass@1 gains on SWE-bench Verified and Terminal-Bench 2.0.","arXiv :2607 .05378v 1 [ cs .LG] 6 Jul 2026  \nCompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents  \nYujiang Li*† Zhenyu Hou*† Yi Jing† Jie Tang Yuxiao Dong  \nTsinghua University  \nAbstract  \nLong-horizon agentic LLMs are increasingly limited by finite context windows, as extended interaction trajectories can exceed the maximum context length before atask is completed. Context compaction offers a natural solution by summarizing previous interaction states and continuing the rollout under a compressed context, but incorporating compaction into reinforcement learning remains underexplored.  \nWe propose CompactionRL, a reinforcement learning strategy to train long-horizon agentic LLMs with context compaction. Our approach jointly optimizes task execution and summary generation with token-level loss normalization and crosstrajectory generalized advantage estimation. This design enables the LLM agents to learn from compacted long-horizon trajectories. We train CompactionRL on top of open models and observe consistent performance gains on agentic coding tasks.  \nCompactionRL enables the open GLM-4.5-Air model (106B-A30B) to achieve  \nPass@1 scores of 66.8% on SWE-bench Verified and 24.5% on Terminal-Bench  \n2.0, with absolute gains of 7.0 and 3.1 points, respectively. Built upon GLM-4.7-Flash (30B-A3B), CompactionRL improves Pass@1 by 5.5 and 6.8 points, reaching 56.0% on SWE-bench Verified and 20.2% on Terminal-Bench 2.0, respectively.  \nCompactionRL is thus deployed in the RL pipeline for training the open GLM-5.2 model (750B-A40B) .  \n1 Introduction  \nLarge language model (LLM) agents are increasingly applied to long-horizon interactive tasks, such as software engineering, terminal-based problem solving, and web interaction [Yao et al., 2023, Nakano et al., 2021, Jimenez et al., 2023] . These tasks require agents to repeatedly reason, act, observe environment feedback, and revise their plans over many steps. As the interaction proceeds, the accumulated history, including tool outputs, intermediate reasoning, error messages, and partial solutions, can exceed the model’s finite context window. Although longer-context models alleviate this issue, scaling context length alone is costly and does not fully solve degraded utilization overlong sequences [Beltagy et al., 2020, Dao et al., 2022, Liu et al., 2024, Hsieh et al., 2024] . Thus, long-horizon RL for LLM agents requires mechanisms that allow training to proceed under a fixed context budget.  \nContext compaction provides a natural way to address this limitation. When the history approaches the context limit, earlier interaction history can be summarized into a shorter state, after which the agent resumes from the compacted summary and a small amount of recent context. Related ideas have been explored in language-agent memory, reflection, and long-horizon interaction systems [Shinnet al., 2023, Park et al., 2023, Sun et al., 2025] . However, prior uses of compaction are mostly treated  \nas inference-time heuristics or external memory operations. In RL training, compaction has a more *Equal Contribution.  \n†Work done while YL, ZH, and YJ interned at Z.AI.  \nPreprint.  \n| Accuracy (%) | 100\u003Cbr>80\u003Cbr>60\u003Cbr>40\u003Cbr>20\u003Cbr>0 |  GLM-4.7-Flash  GLM-4.7-Flash + CompactionRL\u003Cbr> GLM-4 . 5-Air   GLM-4.5-Air + CompactionRL +7.0 +5.5 +6.8\u003Cbr>+3.1\u003Cbr>\u003Cbr>\u003Cbr>30B 106B 30B 106B\u003Cbr>SWE-Verified Terminal-Bench 2.0 |\n| --- | --- | --- |\n\nFigure 1: Left: Context compaction allows execution to continue under a fixed context window by initiating a new trace from a compressed summary once the context budget is exhausted. Without compaction, execution terminates upon exhaustion of the context budget. Right: CompactionRL consistently outperforms baseline models on SWE-bench Verified and Terminal-Bench 2.0 under compaction evaluation at both GLM-4.7-Flash(30B-A3B) and GLM-4.5-Air(106B-A30B) scales.  \nfundamental role: once a summary replaces the original history, it determines w","cbCaiiWGpYsDmSqJ","https://ap.wps.com/l/cbCaiiWGpYsDmSqJ","pdf",566292,1,13,"English","en",105,"# Introduction\n## Problem: context limits for long-horizon agents\n## Background: existing RL and compaction approaches\n## Proposed method: CompactionRL","[{\"question\":\"What problem does CompactionRL address for long-horizon LLM agents?\",\"answer\":\"It addresses the finite context window limit, where long interaction histories can exceed maximum context length before the task finishes.\"},{\"question\":\"How does context compaction help during agent rollout?\",\"answer\":\"When the history approaches the context limit, earlier interaction history is summarized into a shorter state, and the agent continues from that compact summary with a small amount of recent context.\"},{\"question\":\"What is the key idea behind CompactionRL’s training strategy?\",\"answer\":\"CompactionRL jointly optimizes task execution actions and summarization actions under the same final task reward, enabling RL training to extend beyond peak context length while keeping a fixed working context budget.\"}]",1784191579,33,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"compactionrl-reinforcement-learning-with-context-compaction-for-long-horizon-agents","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/compactionrl-reinforcement-learning-with-context-compaction-for-long-horizon-agents/83942/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does CompactionRL address for long-horizon LLM agents?","Question",{"text":74,"@type":75},"It addresses the finite context window limit, where long interaction histories can exceed maximum context length before the task finishes.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does context compaction help during agent rollout?",{"text":79,"@type":75},"When the history approaches the context limit, earlier interaction history is summarized into a shorter state, and the agent continues from that compact summary with a small amount of recent context.",{"name":81,"@type":72,"acceptedAnswer":82},"What is the key idea behind CompactionRL’s training strategy?",{"text":83,"@type":75},"CompactionRL jointly optimizes task execution actions and summarization actions under the same final task reward, enabling RL training to extend beyond peak context length while keeping a fixed working context budget.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]