[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85182-en":3,"doc-seo-85182-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85182,962075114765,"Quinn","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation","Automated chest X-ray report generation benefits from reinforcement learning and large language models, yet RL training often becomes unstable due to fixed KL regularization and a static reference policy that accumulates KL pressure over time. REVA-PO introduces Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR) to stabilize long-horizon optimization. RER reweights per-response KL penalties using advantage and reference-policy entropy, while VAPR periodically synchronizes reference and current policies to the best validation checkpoint, expanding viable exploration space. Experiments on MIMIC-CXR and IU-Xray show new SOTA results in linguistic quality and clinical accuracy.","arXiv :2607 . 10147v1 [ cs .CV] 11 Jul 2026  \nREVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation  \nLi Guo, Anas M. Tahir, and Z. Jane Wang  \nUniversity of British Columbia, Vancouver, BC, Canada [lguo@ece.ubc.ca](lguo@ece.ubc.ca)  \nAccepted to ECCV 2026  \nAbstract. Automated chest X-ray report generation has recently benefited from reinforcement learning (RL) and large language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback-Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We propose ResponseWeighted and Validation-Anchored Policy Optimization (REVA-PO), a RL framework that stabilizes long-term training via Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR) .  \nRER dynamically adjusts per-response KL weights based on advantage and reference-policy entropy, relaxing constraints for high-quality responses while tightening them for low-quality ones. Complementarily, VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure to expand the viable exploration space. To ensure a robust starting point, we employ a three-stage pipeline consisting of warm-up training, classifier-guided supervised fine-tuning, and RL. Extensive evaluationson MIMIC-CXR and IU-Xray demonstrate that REVA-PO sets new state-of-the-art benchmarks in both linguistic quality and clinical accuracy. Notably, BLEU-4 improves by 5 . 1% on MIMIC-CXR and 3 .6% on IU-Xray, while CheXpert F1 and RadGraph F1 scores increase by 4.5% and 12 .8%, respectively, over prior leading methods. The code is publicly available at [https://github.com/LiGuo12/REVA_PO/](https://github.com/LiGuo12/REVA_PO/) .  \nKeywords: Reinforcement Learning · KL Regularization · Chest X-ray Report Generation  \n1 Introduction  \nChest X-rays (CXR) are essential for diagnosing and monitoring chest organ conditions, yet the demand for interpretation often exceeds radiologist capacity, increasing turnaround times and diagnostic risks [4, 22, 36] . While automated CXR report generation (CXR-RG) has progressed significantly [13,41], generated reports still struggle with nuanced word choice and sentence fluency compared to professional reports.  \n2 L. Guo et al.  \nTo improve report quality, recent work has integrated large language models (LLMs) [18,26,43,45,58] . Standard approaches often utilize contrastive learning to map radiographic features into LLM embedding space—a process that is memoryintensive and prone to overfitting on limited medical datasets. While the emerging large vision-language models (LVLMs) pretrained on broad image–text data already align visual and textual spaces [1, 11, 53] . Inspired by the advantages of LVLMs, we propose adopting a pretrained LVLM for CXR-RG, which eliminates the need to train a separate image encoder and reduces memory use while mitigating overfitting.  \nFrom a training perspective, reinforcement learning from human feedback (RLHF) has demonstrated superior performance in text generation [2, 30, 67] . Standard proximal policy optimization (PPO) [37] requires a critic model often as large as the policy itself. To optimize resource use, we adopt group relative policy optimization (GRPO) [12, 38], which omits the critic by estimating baselines from multiple responses per prompt. Furthermore, training a reward model requires labeled human-preference data, which is costly to collect from radiologists. Following previous RL work in CXR-RG [20, 32], we utilize the BLEU-4 score between the generated and reference reports as the reward, thereby avoiding the need for a learned reward model.  \nWhile Kullback–Leibler (KL) regularization is the standard safeguard against RL instability, a fixed KL coefficient β often creates a “lose-lose” scenario: it either stifles productive exploration or fails to prevent destabilizing update","cbCaihGV1FLShsdK","https://ap.wps.com/l/cbCaihGV1FLShsdK","pdf",2588158,5,1,19,"English","en",105,"# Introduction\n## Integrating Vision-Language Models for CXR Report Generation\n## RLHF and Efficient Policy Optimization (GRPO)\n## Limitations of Fixed KL Regularization and Instance/Batch Adaptation\n## Proposed Method: Response-Weighted Regularization (RER)\n## Proposed Method: Validation-Anchored Policy Reset (VAPR)\n## Reward Design Without Learned Reward Models","[{\"question\":\"What problem does REVA-PO address in reinforcement learning for chest X-ray report generation?\",\"answer\":\"REVA-PO targets RL instability and limited exploration caused by fixed KL regularization and a static reference policy that accumulates KL pressure over time.\"},{\"question\":\"How does Response-Weighted Regularization (RER) improve training stability?\",\"answer\":\"RER adjusts per-response KL penalty weights using advantage and reference-policy entropy, relaxing constraints for high-quality responses and tightening them for low-quality ones.\"},{\"question\":\"How does Validation-Anchored Policy Reset (VAPR) prevent reference drift and KL domination?\",\"answer\":\"VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure and reducing ineffective KL loss due to outdated references.\"}]",1784201592,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"reva-po-stabilizing-reinforcement-learning-for-chest-x-ray-report-generation","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/reva-po-stabilizing-reinforcement-learning-for-chest-x-ray-report-generation/85182/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does REVA-PO address in reinforcement learning for chest X-ray report generation?","Question",{"text":76,"@type":77},"REVA-PO targets RL instability and limited exploration caused by fixed KL regularization and a static reference policy that accumulates KL pressure over time.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does Response-Weighted Regularization (RER) improve training stability?",{"text":81,"@type":77},"RER adjusts per-response KL penalty weights using advantage and reference-policy entropy, relaxing constraints for high-quality responses and tightening them for low-quality ones.",{"name":83,"@type":74,"acceptedAnswer":84},"How does Validation-Anchored Policy Reset (VAPR) prevent reference drift and KL domination?",{"text":85,"@type":77},"VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure and reducing ineffective KL loss due to outdated references.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":20,"slug":137},"General","general"]