[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84989-en":3,"doc-seo-84989-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84989,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","MMAgent-R2 Learning to Rerank and Reject for Agentic mRAG","Knowledge-based Visual Question Answering (KB-VQA) depends on retrieving the correct visual entity from large-scale encyclopedic knowledge bases and generating answers grounded in that entity. Existing multimodal mRAG systems often match candidates using global visual features, causing confusion among visually similar but factually mismatched entries; subsequent filtering remains limited to a fixed candidate pool, so retrieval errors propagate. MMAgent-R2 introduces agentic mRAG with visual reranking and active rejection, plus step-level verification rewards optimized via GRPO training, improving performance on difficult retrieval and multi-hop reasoning benchmarks.","arXiv :2607 .07383v 1 [ cs .CV] 8 Jul 2026  \nMMAgent-R2 : Learning to Rerank and Reject for  \nAgentic mRAG  \nTao Zhang∗ 1 ,2 ,3 ,4, Ziqi Zhang∗ 1 ,3, Zongyang Ma 1 ,3, Yuxin Yang 1 ,2 ,3, Bing Li†1 ,3 ,5, Chunfeng Yuan 1 ,3, Kang Rong4, Fengyun Rao4, Jing LYU4, and Weiming Hu 1 ,2 ,3 ,6  \n1 State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA  \n{zhangtao2023,mazongyang2020,[yangyuxin2023}@ia.ac.cn](yangyuxin2023}@ia.ac.cn)[ ](yangyuxin2023}@ia.ac.cn){ziqi.zhang,bli,cfyuan,[wmhu}@nlpr.ia.ac.cn](wmhu}@nlpr.ia.ac.cn)  \n2 School of Artificial Intelligence, University of Chinese Academy of Sciences  \n3 Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information  \n4 WeChat Vision, Tencent Inc.  \n{rickrong,fengyunrao,[eckolv}@tencent.com](eckolv}@tencent.com)  \n5 PeopleAI Inc.  \n6 School of Information Science and Technology, ShanghaiTech University  \n∗ Equal contribution †Corresponding author  \nAbstract. Knowledge-based Visual Question Answering (KB-VQA) requires models to retrieve visual entities matching the query image from large-scale encyclopedic knowledge bases and answer related questions.  \nExisting multimodal Retrieval Augmented Generation (mRAG) methods rely on global visual features to match candidate entities, yet when the knowledge base contains numerous visually similar entities, the retriever struggles to distinguish them, populating the candidate set with visually similar but factually mismatched distractors. Since subsequent processing steps such as noise filtering are also confined to this fixed candidate set, errors from failed retrieval inevitably propagate to the final answer. To address these challenges, we propose MMAgent-R2 , anagentic mRAG framework that integrates visual reranking and active rejection as its internal verification mechanism. Visual reranking directly compares query and candidate images, capturing discriminative details beyond textual descriptions to precisely identify the target entity among similar candidates; active rejection discards unreliable results and retrieves additional candidates when no confident match is found, moving beyond the fixed candidate pool. We design a composite reward function with step-level verification rewards and achieve joint optimization of external retrieval, internal verification, and answer generation via GRPO training. Experiments on InfoSeek, E-VQA, and MMhops demonstrate that MMAgent-R2 achieves state-of-the-art performance, with particularly notable advantages in challenging retrieval scenarios and complex multi-image multi-hop reasoning tasks.  \nKeywords: Knowledge-based VQA · Retrieval-Augmented Generation  \n· Visual Agent · Reinforcement Learning  \n2 T. Zhang et al.  \nFig. 1: Comparison between the Retrieve-then-Postprocess mRAG and MMAgent-R2 .(a) With a fixed candidate set retrieved via global visual features, the model can be misled by visually similar yet mismatched entries. (b) MMAgent-R2 compares the query image against candidates (①–⑧) to rerank them, and rejects unreliable results to retrieve additional candidates when no confident match is found, enabling correct entity identification and answering.  \n1 Introduction  \nKnowledge-based Visual Question Answering (KB-VQA) [6,20,35] requires models to recognize visual entities in images and answer questions about their attributes. As illustrated in Fig. 1, answering ”When was this building constructed?”hinges on precisely identifying the building and subsequently associating it with external knowledge (its construction year) . Recently, multimodal Retrieval Augmented Generation (mRAG) [5,8,10,12,18,21,31,34,36] methods have emerged asa promising approach for such tasks. These methods generally adopt a \"Retrievethen-Postprocess\" paradigm, where the retrieval stage attempts to locate candidate entries visually similar to the query entity from a large-scale visual-textual encyclopedia knowledge base, and the post-processing stage then filters from a small, fixe","cbCaienHZWSWU9MI","https://ap.wps.com/l/cbCaienHZWSWU9MI","pdf",8495533,1,21,"English","en",105,"# Abstract\n# Introduction\n## Problem: identification bottleneck in fixed candidate retrieval\n## Proposed solution: MMAgent-R2 with visual reranking and active rejection","[{\"question\":\"What challenge do existing multimodal mRAG methods face in KB-VQA?\",\"answer\":\"They struggle to distinguish visually similar entities during retrieval, which introduces distractors. Because later steps work only on a fixed candidate pool, retrieval mistakes propagate into the final answer.\"},{\"question\":\"How does MMAgent-R2 improve candidate selection?\",\"answer\":\"It reranks candidates by directly comparing the query image with candidate images to capture discriminative visual details beyond textual descriptions.\"},{\"question\":\"What does active rejection do in MMAgent-R2?\",\"answer\":\"When no confident match exists, it discards unreliable results and retrieves additional candidates rather than being constrained to the initial candidate set.\"}]",1784200070,53,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"mmagent-r2-learning-to-rerank-and-reject-for-agentic-mrag","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/mmagent-r2-learning-to-rerank-and-reject-for-agentic-mrag/84989/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What challenge do existing multimodal mRAG methods face in KB-VQA?","Question",{"text":74,"@type":75},"They struggle to distinguish visually similar entities during retrieval, which introduces distractors. Because later steps work only on a fixed candidate pool, retrieval mistakes propagate into the final answer.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does MMAgent-R2 improve candidate selection?",{"text":79,"@type":75},"It reranks candidates by directly comparing the query image with candidate images to capture discriminative visual details beyond textual descriptions.",{"name":81,"@type":72,"acceptedAnswer":82},"What does active rejection do in MMAgent-R2?",{"text":83,"@type":75},"When no confident match exists, it discards unreliable results and retrieves additional candidates rather than being constrained to the initial candidate set.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]