[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83502-en":3,"doc-seo-83502-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83502,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782698725881665579",8,"Research & Report","Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization","Scientific reasoning matters for large language models, but improving training robustness and efficiency for multi-step reasoning remains open. The work studies instruction-based molecular optimization, where answer-only SFT collapses reasoning and RLVR suffers sparse feedback from invalid or un-similar candidates. Reference-guided Policy Optimization helps but caps performance when references are weak or misaligned. Active-GRPO adaptively decides per instance when to imitate and when to reinforce, while upgrading references during training. Experiments on TOMG-Bench MOLOPT show statistically significant gains across LogP, MR, and QED.","arXiv :2607 .0053 1v 1 [ cs .LG] 1 Jul 2026  \nActive-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization  \nXuefeng Liu1∗, Mingxuan Cao2∗, Qinan Huang3 , Thomas Brettin5 , Rick L. Stevens4,5 , Le Cong1  \n1 School of Medicine, Stanford University  \n2Data Science Institute, University of Chicago  \n3Pritzker School of Molecular Engineering, University of Chicago  \n4Department of Computer Science, University of Chicago  \n5Argonne National Laboratory  \nAbstract  \nScientific reasoning is an increasingly important capability of large language mod  \nels, yet improving the robustness and efficiency of training such reasoning remains  \na key open challenge. We study this problem in instruction-based molecular op  \ntimization, where answer-only supervised fine-tuning (SFT) collapses multi-step  \nreasoning and reinforcement learning with verifiable rewards (RLVR) suffers from  \nsparse feedback. Reference-guided Policy Optimization (RePO) mitigates both  \nby anchoring policy updates to dataset-provided references, but its effectiveness  \nis tightly coupled to reference quality: weak or misaligned references impose a  \nperformance ceiling. To overcome this ceiling, we propose active reasoning, a  \nparadigm in which the policy actively decides, on a per-instance basis, when to  \nimitate a reference and when to reinforce its own discoveries, while continuously  \nupgrading what it imitates. We instantiate this paradigm as Active Group Relative  \nPolicy Optimization (Active-GRPO), realized through two coupled mechanisms:  \nactive imitate-reinforce and active referencing. The former performs imitation  \nlearning when the reference still outperforms the policy’s own candidates, and  \nshifts to self-improvement via reinforcement learning once the policy has gener  \nated molecules that surpass the reference. The latter continuously upgrades the  \nreference itself by replacing it with the best policy-generated candidate discovered  \nso far, progressively raising the imitation target and ensuring that reference guid  \nance remains informative—rather than restrictive—throughout training. Across  \nTOMG-Bench MOLOPT, Active-GRPO improves average SR×Sim from 0.0959  \nfor GRPO and 0.1665 for RePO to 0.1773 under matched three-seed evaluation,  \nwith statistically significant gains on LogP, MR, and QED.  \n1 Introduction  \nLarge language models (LLMs) have rapidly emerged as general-purpose reasoning engines, demonstrating strong performance on tasks that demand multi-step deliberation rather than surface pattern matching [17, 38] . Through advances in chain-of-thought prompting [52], supervised fine-tuning (SFT) on reasoning traces [36, 56], and reinforcement learning with verifiable rewards (RLVR)[19, 28], modern LLMs can now solve competition-level mathematics [11, 20], write and debug complex code [9, 24], and conduct structured analyses across diverse domains. This progress has motivated a growing line of work that brings LLM reasoning to bear on scientific discovery [1, 51], where success often hinges on navigating combinatorially large hypothesis spaces under domain-specific constraints. From hypothesis generation and experimental design to candidate screening in chemistry, biology, and materials science [7, 23, 49], LLMs are increasingly positioned not as passive question  \n∗Equal Contribution. Correspondence to: Xuefeng Liu \u003Cxfl@stanford.edu>, Mingxuan Cao \u003Cmcao@uchicago.edu>  \nanswerers but as active reasoners that propose, evaluate, and refine scientific artifacts. Yet making such reasoning robust and sample-efficient to train remains a central open challenge—particularly in scientific domains where outputs must satisfy strict, programmatically verifiable constraints.  \nAmong these scientific reasoning tasks, instruction-based molecular optimization has emerged as a particularly demanding testbed [29, 31] . Given a source molecule and a natural-language instruction specifying desired property changes—for example, impro","cbCaiivSZeKR8oh4","https://ap.wps.com/l/cbCaiivSZeKR8oh4","pdf",2087539,1,25,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does Active-GRPO address in instruction-based molecular optimization?\",\"answer\":\"It targets fragile multi-step scientific reasoning training, where SFT collapses reasoning and RLVR provides sparse feedback under strict validity and similarity constraints.\"},{\"question\":\"How does Active-GRPO decide between imitation and reinforcement during training?\",\"answer\":\"Active imitate-reinforce performs imitation learning when the reference is still better than the policy candidates, then switches to reinforcement learning once policy-generated molecules surpass the reference.\"},{\"question\":\"How does Active-GRPO improve the references used for guidance?\",\"answer\":\"Active referencing continuously replaces the reference with the best policy-generated candidate found so far, raising the imitation target and keeping guidance informative rather than restrictive.\"}]",1784188472,63,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"active-grpo-adaptive-imitation-and-self-improving-reasoning-for-molecular-optimization","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/active-grpo-adaptive-imitation-and-self-improving-reasoning-for-molecular-optimization/83502/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does Active-GRPO address in instruction-based molecular optimization?","Question",{"text":75,"@type":76},"It targets fragile multi-step scientific reasoning training, where SFT collapses reasoning and RLVR provides sparse feedback under strict validity and similarity constraints.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Active-GRPO decide between imitation and reinforcement during training?",{"text":80,"@type":76},"Active imitate-reinforce performs imitation learning when the reference is still better than the policy candidates, then switches to reinforcement learning once policy-generated molecules surpass the reference.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Active-GRPO improve the references used for guidance?",{"text":84,"@type":76},"Active referencing continuously replaces the reference with the best policy-generated candidate found so far, raising the imitation target and keeping guidance informative rather than restrictive.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]