[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-273628-105":59,"doc-detail-273628-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","spade-self-play-in-adaptive-synthetic-executable-environments","SPADE♠ - Self-Play in Adaptive Synthetic Executable Environments","","Continuous self-improvement depends on expanding pools of self-generated, diverse, adaptive goals. For language agents, prior environment pools remain fixed as learners scale, limiting further progress. SPADE introduces an RL self-play framework where a single LLM designs executable long-horizon environments and a reasoning agent learns to solve them through a shared stateful interface. The designer optimizes hint-based regret to target the agent’s capability boundary while preserving feasibility. Experiments show consistent gains on multiple held-out math, science, code, and reasoning benchmarks, with larger tool-use margins at scale.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/spade-self-play-in-adaptive-synthetic-executable-environments/273628/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/spade-self-play-in-adaptive-synthetic-executable-environments/273628.png","ImageObject",300,407,{"name":92,"@type":93},"Theodore","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-28","2026-09-15",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",16,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What does SPADE do in adaptive self-play training?","Question",{"text":113,"@type":114},"SPADE is an RL self-play framework where one LLM plays two roles: an Environment Designer that writes executable, long-horizon training environments, and a Reasoning Agent that learns to solve them.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does SPADE guide the Environment Designer to improve the agent?",{"text":118,"@type":114},"It estimates the Reasoning Agent’s regret using the reward gap with and without privileged hints, then trains the Environment Designer to target environments near the agent’s capability edge while remaining feasible.",{"name":120,"@type":111,"acceptedAnswer":121},"What performance improvements does SPADE report?",{"text":122,"@type":114},"Across held-out math, science, code, and reasoning benchmarks, SPADE improves over the strongest fixed-environment baseline by an average of +5.3, and it shows larger gains for tool-use settings such as BFCL v4 multi-turn and ACEBench-Agent.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},273628,1789472311,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},7971461740886,"https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2","arXiv : 2608 . 19 197v 3 [cs .CL] 3 1 Aug 2026  \nSPADE♠: Self-Play in Adaptive Synthetic Executable Environments  \nBo Liu 1 ,2 ,∗ , Simon Yu3 ,∗ , Yiding Jiang4 , Ao Qu5 , Andrew Zhao , Zichen Liu6 , Junsu Kim7 , Zijian Zhou6 , Seungone Kim4 , Tongzheng Ren , Mickel Liu 1 , Hanfei Yu8 , Zhaorun Chen9 , Weiyan Shi3 , Paul Pu Liang5 , Luke Zettlemoyer 1 , Yejin Choi2 ,†, Natasha Jaques 1 ,†  \n1 University of Washington, 2 Stanford University, 3 Northeastern University, 4 Carnegie Mellon University, 5 Massachusetts Institute of Technology, 6 National University of Singapore, 7 Seoul National University, 8 Stevens Institute of Technology, 9 University of Chicago  \n∗ Equal contribution, †Joint last author  \nContinuous self-improvement requires an ever-expanding pool of self-generated, diverse, adaptive goals. For language agents, existing training environment pools (hand-curated, statically synthesized, or frozen-verifier) keep the goal distribution fixed as the learner scales. We introduce SPADE (Self-Play in Adaptive Synthetic Executable Environments), a self-play RL framework in which a single LLM plays two roles: an Environment Designer that writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface, and a Reasoning Agent that learns to act in them. Each is a stateful, multi-turn environment (state transitions, reward functions, and verification code), so one interface spans reasoning problems and multi-step agentic tool use. The Reasoning Agent’s regret is estimated using the gap between its reward with and without privileged hints; in optimizing this regret signal the Environment Designer learns to target environments atthe edge of the agent’s capabilities while keeping them feasible. Through extensive experimentation, we find several components critical to success: grounding the Environment Designer on documents sampled from a large pretraining corpus, and giving it an accumulated environment memory. Scaling to 30B-parameter models, SPADE improves over the strongest fixed-environment baseline by +5 .3 on average across eight held-out math, science, code, and reasoning benchmarks, and lifts the tool-use setting by +5 .7 on BFCL v4 multi-turn and +13 .9 on ACEBench-Agent; on the games setting, the margin over the strongest baseline grows with model scale. By making environment design itself alearnable component, SPADE takes a concrete step toward open-ended self-improvement.  \nCorrespondence: [benjaminliu.eecs@gmail.com](benjaminliu.eecs@gmail.com), [yejinc@stanford.edu](yejinc@stanford.edu), [nj@cs.washington.edu](nj@cs.washington.edu)  \nCode: [https://github.com/spade-rl/spade](https://github.com/spade-rl/spade)  \nProject: [https://spade-rl.github.io](https://spade-rl.github.io)  \nAvg. improvement over base (%)  \n25  \n20  \n15  \n10  \n5  \n0  \nGames (30B-A3B): transfer to 8 OOD evals  \n Fixed-envGRPO  Fixed-env RLVE  SPADE  \n0 100 200 300 400  \nTraining step  \nImprovement over base (%)  \nTool use (30B-A3B): transfer to 3 evals  \n2-bench  BFCLv4 MT  ACEBench-Agent  \n0 100 200 300 400  \nTraining step  \nFigure 1 SPADE designs and solves its own training environments in both settings. Left: a single LLM πθ plays both roles, an Environment Designer that writes an executable environment e with a privileged hint h, and a Reasoning Agent that solves e with and without h; the return gap rewards the Environment Designer (hint-based regret), task completion rewards the Reasoning Agent, and both update the same weights. Middle: average relative improvement over the untrained base across the eight games-setting evals ((score − base)/base), versus Fixed-env RLVE (orange) and Fixed-env GRPO (gray) . Right: per-benchmark relative improvement of SPADE-30B-A3B on the three tool-use evals (τ2-bench, BFCL v4 multi-turn, and ACEBench-Agent; Table 2) .  \n1 Introduction  \nAgentic AI has become broadly capable: language models now reason over long horizons, use tools, search the web, and op","cbCairCrxawUzfQE","https://ap.wps.com/l/cbCairCrxawUzfQE","pdf",2995572,73,"English","# Introduction\n## Motivation and problem setting\n## SPADE framework and method\n## Experimental results and benchmarks\n## Key findings","[{\"question\":\"What does SPADE do in adaptive self-play training?\",\"answer\":\"SPADE is an RL self-play framework where one LLM plays two roles: an Environment Designer that writes executable, long-horizon training environments, and a Reasoning Agent that learns to solve them.\"},{\"question\":\"How does SPADE guide the Environment Designer to improve the agent?\",\"answer\":\"It estimates the Reasoning Agent’s regret using the reward gap with and without privileged hints, then trains the Environment Designer to target environments near the agent’s capability edge while remaining feasible.\"},{\"question\":\"What performance improvements does SPADE report?\",\"answer\":\"Across held-out math, science, code, and reasoning benchmarks, SPADE improves over the strongest fixed-environment baseline by an average of +5.3, and it shows larger gains for tool-use settings such as BFCL v4 multi-turn and ACEBench-Agent.\"}]","SPADE♠ - Self-Play in Adaptive Synthetic Executable Environments | PDF",184]