[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84034-en":3,"doc-seo-84034-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84034,13056703019404,"Miles","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Contextual Multimodal Document Retrieval (CMDR)","Multimodal document retrieval seeks to return relevant pages while preserving both textual and visual evidence from original documents. Existing benchmarks mainly test lexical or semantic matching and encode pages independently, ignoring cross-page context needed for queries that combine information across multiple pages. The paper introduces CMDR and CMDR-Bench, a contextual multimodal retrieval task and benchmark, and proposes CMDR-Embed with a jointly contextualized embedding scheme plus CMCL contrastive learning to balance context modeling and page-level discriminability.","arXiv :2607 .05927v 1 [ cs .IR] 7 Jul 2026  \nCMDR:  \nContextual Multimodal Document Retrieval  \nRyota Tanaka, Taku Hasegawa, and Kyosuke Nishida  \nHuman Informatics Labs., NTT, Inc., Tokyo, Japan {ryota.tanaka,taku.hasegawa,[kyosuke.nishida}@ntt.com](kyosuke.nishida}@ntt.com)  \nAbstract. Multimodal document retrieval aims to retrieve relevant pages while preserving both textual and visual content from the original document. However, existing benchmarks primarily evaluate simple lexical or semantic matching, and most methods encode pages independently.  \nConsequently, they overlook the contextual information in the document required to resolve queries that aggregate information across multiple pages. In this paper, we introduce CMDR and CMDR-Bench, a new multimodal document retrieval task and benchmark that require modeling document context. To address this challenge, we propose CMDR-Embed, a contextual multimodal embedding framework that explicitly incorporates document context by jointly encoding multiple pages and deriving page-level embeddings from a shared contextual representation. Furthermore, we introduce CMCL, a contextual multimodal contrastive learning objective for effectively training CMDR-Embed, which balances contextual modeling with page-level discriminability. Experiments demonstrate that CMDR-Embed significantly outperforms non-contextual embeddings, highlighting the importance of context-aware multimodal embeddings for advancing document retrieval 1 .  \nKeywords: Document Retrieval · Embeddings  \n1 Introduction  \nReal-world documents are inherently multimodal; they contain both textual and visual elements, with their content distributed across different regions. The ability to accurately retrieve information relevant to a user query from such diverse documents is crucial for various applications such as search engines [5, 31] and Retrieval-Augmented Generation (RAG) [19, 27] . Unlike traditional text-based retrieval methods [21,26,45], which focus on textual data, multimodal document retrieval [17, 37, 47, 48, 61] has emerged as a promising approach that seamlessly integrates multimodal information by treating documents as images.  \nHowever, as illustrated in Figure 1a, existing multimodal document retrieval benchmarks [6, 13, 37, 48] primarily evaluate direct retrieval using simple lexical or semantic matching between queries and individual pages. As a result, they do not assess retrieval capabilities that leverage document context, such as 1 􀂀 Project Page: [https://cmdr-bench.github](https://cmdr-bench.github).io  \n2 R. Tanaka et al.  \n\\#􀂗Üã􀂁¼ã SØ«Ã􀂍«Õ¼􀂗Ü Ê¡ q«Üè􀂁¼ S􀂗Ø􀂍􀂗Õã«Êà  \n*HVWDOW SV\\FKRORJ\\ ZDV FRQFHLYHG LQ WKH %HUOLQ 6FKRRO RI ([SHULPHQWDO 3V\\FKRORJ\\􀀏 DQG WULHV WR XQGHUVWDQG WKH ODZV RI RXU DELOLW\\ WR DFTXLUH DQG PDLQWDLQ PHDQLQJIXO SHUFHSWLRQV􀀑  \nƔ 􀀋*HUPDQ􀀝 *HVWDOW>ܳԥ ݕޖWDOW@ 􀀅VKDSH􀀏 IRUP􀀅􀀌  \nPage 1  \nPage 2  \nPage N  \nEncode Independently  \nNon-Contextual  \nEmbedding  \nContextual  \nEmbedding  \nExisting Methods (e.g., ColPali)  \nOur Method: CMDR-Embed  \n Power button  \nPress this button to power on or o! the system.  \n…  \n…  \n…  \n…  \nFig. 1: Comparison benchmarks and methods. Unlike previous benchmarks that rely on direct keyword or simple semantic matching, CMDR-Bench focuses on indirectly retrieving relevant pages by leveraging document context. In contrast to non-contextual embeddings that encode each page independently, CMDR-Embed jointly encodes multiple pages to capture cross-page contextual relationships.  \ncross-page dependencies and document global structure. Moreover, existing retrieval methods [17, 32, 35, 47, 61] independently encode each page in multi-page documents, implicitly assuming that contextualization yields negligible benefit. In practice, however, important information is often distributed across pages, requiring models to reason over multiple pages to fully resolve a query.  \nMotivated by these limitations, we propose a new multimodal document retrieval task, Contextual Mult","cbCaiideznsItpWF","https://ap.wps.com/l/cbCaiideznsItpWF","pdf",9385493,4,1,29,"English","en",105,"# Introduction\n## Motivation and limitations of existing benchmarks\n## CMDR task and CMDR-Bench benchmark\n## Proposed method: CMDR-Embed and CMCL","[{\"question\":\"What problem does CMDR address in multimodal document retrieval?\",\"answer\":\"CMDR targets retrieval settings where important information is distributed across multiple pages, requiring the model to use document context rather than matching queries to individual pages independently.\"},{\"question\":\"What are CMDR and CMDR-Bench?\",\"answer\":\"CMDR defines a contextual multimodal document retrieval task, and CMDR-Bench provides a new benchmark with human-annotated queries designed to require multi-page reasoning across multimodal documents.\"},{\"question\":\"How does CMDR-Embed incorporate context and improve retrieval quality?\",\"answer\":\"CMDR-Embed jointly encodes multiple pages to build a shared contextual representation, then derives page-level embeddings from it. It is trained with CMCL to maintain page-level discriminability while leveraging context-aware hard negatives.\"}]",1784192164,73,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"contextual-multimodal-document-retrieval-cmdr","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/contextual-multimodal-document-retrieval-cmdr/84034/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-26","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does CMDR address in multimodal document retrieval?","Question",{"text":75,"@type":76},"CMDR targets retrieval settings where important information is distributed across multiple pages, requiring the model to use document context rather than matching queries to individual pages independently.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are CMDR and CMDR-Bench?",{"text":80,"@type":76},"CMDR defines a contextual multimodal document retrieval task, and CMDR-Bench provides a new benchmark with human-annotated queries designed to require multi-page reasoning across multimodal documents.",{"name":82,"@type":73,"acceptedAnswer":83},"How does CMDR-Embed incorporate context and improve retrieval quality?",{"text":84,"@type":76},"CMDR-Embed jointly encodes multiple pages to build a shared contextual representation, then derives page-level embeddings from it. It is trained with CMCL to maintain page-level discriminability while leveraging context-aware hard negatives.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]