[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86391-en":3,"doc-seo-86391-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},86391,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning","Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models targets self-improving embodied agents that adapt in open-ended, evolving environments. Prior continual learning guidance suggests that naive Sequential Fine-Tuning (Seq. FT) causes catastrophic forgetting, requiring complex CRL strategies. A systematic study of CRL for large pretrained VLAs across lifelong RL benchmarks shows that simple Seq. FT with low-rank adaptation (LoRA) is highly effective, maintaining plasticity, minimizing forgetting, and preserving strong zero-shot generalization, often exceeding advanced CRL methods. Detailed analysis attributes robustness to synergy among large pretraining, parameter-efficient adaptation, and on-policy RL, reshaping the stability–plasticity trade-off for scalable continual adaptation.","arXiv :2603 . 1 1653v 3 [ cs .LG] 11 Jul 2026  \nSimple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning  \nJiaheng Hu 1,∗, Jay Shim 1,∗, Chen Tang2 , Yoonchang Sung3 , Bo Liu 1 , Peter Stone 1,4,†, Roberto Martín-Martín 1,†  \n{jiahengh,jshim1213,pstone, [robertomm}@utexas.edu](robertomm}@utexas.edu)[ ](robertomm}@utexas.edu)1UT Austin 2UCLA 3NTU 4 Sony AI  \n∗ Indicating equal contribution † Indicating equal supervision  \nAbstract  \nContinual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across diverse lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability–plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era.1  \n1 Introduction  \nVision-Language-Action (VLA) models represent an emerging paradigm toward building generalpurpose embodied agents. By fine-tuning VLMs for decision-making, these systems have demonstrated strong generalization across diverse scenarios (O’Neill et al., 2024 ; Kim et al., 2024a ; Black et al., 2024) . However, despite their broad competence, current VLA models remain brittle when deployed in evolving or out-of-distribution settings, where reliability and sustained adaptation become critical. This gap highlights the need for continual learning mechanisms that enable VLAs to incrementally refine and extend their capabilities through ongoing interaction, thereby transforming strong initial generalization into self-sustained, lifelong competence.  \nSuch incremental self-improvement, where an agent needs to learn from a non-stationary stream of tasks and experiences, can be formalized as Continual Reinforcement Learning (CRL) . The simplest approach to tackle CRL is through Sequential Fine-Tuning (Seq. FT), where the model is directly finetuned on each new task or environments as it arrives. However, much prior work has shown that Seq. FT is prone to catastrophic forgetting, where the model’s performance on previously learned tasks degrades substantially as it adapts to new ones (French, 1999 ; Kirkpatrick et al., 2017 ;  \n1 Code is available [at github.com/UT-Austin-RobIn/continual-vla-rl](at github.com/UT-Austin-RobIn/continual-vla-rl).  \nSimple Sequential Fine-Tuning  \nPre-trained VLA Model  \nParam-Efficient Adaptation  \nOn-Policy RL  \nContinual Stream of Tasks  \nLittle Catastrophic  \nPast Tasks  \nTask 1 Task n-1  \nForgetting  \nCurrent Task High Plasticity   \nTask n Task n  \nHigh Zero Shot  \n\n| -\u003Cbr>Generalization Unseen Tasks |  |  |  |\n| --- | --- | --- | --- |\n|  |  |  |  |\n\nFigure 1: VLAs as Natural Continual Learners. We show that the synergy between pre-trained VLA, on-policy RL, and LoRA is enough to overcome catastrophic forgetting while maintaining plasticity, enabling simple Sequential Fine-Tuning to achieve surprisingly good performance.  \nGoodfellow et al., 2013) . To mitigate this effect, existing CRL methods introduce mechanisms such as regularization (Kirkpatrick et al., 2017), replay (Rolnick et al","cbCaikZbrzCrWsG7","https://ap.wps.com/l/cbCaikZbrzCrWsG7","pdf",8237558,5,1,31,"English","en",105,"# Abstract\n# Introduction\n## Motivation: continual learning for VLA agents\n## Sequential Fine-Tuning and catastrophic forgetting\n## Stability–plasticity dilemma in CRL\n## Parameter-efficient adaptation (LoRA) in VLA models\n## Research questions and empirical study","[{\"question\":\"What problem does the paper address for Vision-Language-Action (VLA) agents?\",\"answer\":\"The paper targets the brittleness of current VLA models in evolving or out-of-distribution settings, where reliable and sustained adaptation is needed. It frames incremental self-improvement as Continual Reinforcement Learning (CRL).\"},{\"question\":\"Why does the paper claim simple Sequential Fine-Tuning (Seq. FT) can work for CRL?\",\"answer\":\"Despite conventional expectations of catastrophic forgetting, experiments show that Seq. FT with low-rank adaptation (LoRA) retains high plasticity, shows little to no forgetting, and preserves strong zero-shot generalization. This often outperforms more sophisticated CRL methods.\"},{\"question\":\"What causes the robustness of Seq. FT in their analysis?\",\"answer\":\"The paper attributes robustness to a synergy between the large pretrained VLA model, parameter-efficient adaptation via LoRA, and on-policy reinforcement learning. Together these components reshape the stability–plasticity trade-off, enabling stable and scalable continual adaptation.\"}]",1784211455,78,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"simple-recipe-works-vision-language-action-models-are-natural-continual-learners-with-reinforcement-learning","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/simple-recipe-works-vision-language-action-models-are-natural-continual-learners-with-reinforcement-learning/86391/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-27","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the paper address for Vision-Language-Action (VLA) agents?","Question",{"text":76,"@type":77},"The paper targets the brittleness of current VLA models in evolving or out-of-distribution settings, where reliable and sustained adaptation is needed. It frames incremental self-improvement as Continual Reinforcement Learning (CRL).","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Why does the paper claim simple Sequential Fine-Tuning (Seq. FT) can work for CRL?",{"text":81,"@type":77},"Despite conventional expectations of catastrophic forgetting, experiments show that Seq. FT with low-rank adaptation (LoRA) retains high plasticity, shows little to no forgetting, and preserves strong zero-shot generalization. This often outperforms more sophisticated CRL methods.",{"name":83,"@type":74,"acceptedAnswer":84},"What causes the robustness of Seq. FT in their analysis?",{"text":85,"@type":77},"The paper attributes robustness to a synergy between the large pretrained VLA model, parameter-efficient adaptation via LoRA, and on-policy reinforcement learning. Together these components reshape the stability–plasticity trade-off, enabling stable and scalable continual adaptation.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]