[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85278-en":3,"doc-seo-85278-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85278,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search","Multimodal information is crucial for e-commerce search ranking. Prior approaches fine-tune general multimodal large language models using collaborative signals and reuse the learned representations as basic item features, yet they overlook heterogeneous signals needed for multitask ranking and underutilize multimodal embeddings for user behavior modeling. The proposed MMRM unifies multiple collaborative signals via a shared backbone with task tokens and projection layers, learning multiplex item representations in one pass and deriving multiplex user representations from search-based behavior sequences. Experiments show strong efficiency and effectiveness, and the model has been deployed in JD’s search engine with significant gains for millions of daily users.","MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search  \nZhen-Lin Chen  \nJD.COM Beijing, China [chenzhenlin6@jd.com](chenzhenlin6@jd.com)  \nJianmin Chen  \nJD.COM Beijing, China [chenjianmin.23@jd.com](chenjianmin.23@jd.com)  \nMaosen Sheng  \nJD.COM Beijing, China [shengmaosen.1@jd.com](shengmaosen.1@jd.com)  \nZhuojian Xiao  \nJD.COM Beijing, China [xiaozhuojian5@jd.com](xiaozhuojian5@jd.com)  \nPeng Lin  \nJD.COM Beijing, China [linpeng47@jd.com](linpeng47@jd.com)  \nDongyue Wang  \nJD.COM Beijing, China [wangdongyue@jd.com](wangdongyue@jd.com)  \nXiwei Zhao  \nJD.COM Beijing, China [zhaoxiwei@jd.com](zhaoxiwei@jd.com)  \narXiv :2607 . 1 1030v 1 [ cs .IR] 13 Jul 2026  \nAbstract  \nMultimodal information is pivotal for e-commerce search ranking. Existing works leverage multimodal data typically by fine-tuning general Multimodal Large Language Models (MLLMs) via collaborative signals, subsequently integrating the derived representations into ranking models as item features. Despite their efficacy, these methods face two primary limitations: (1) they rely on a single collaborative signal for MLLM fine-tuning, failing to exploit the heterogeneous signals essential for multitask ranking; and (2) they treat multimodal representations as regular item features in ranking models, underutilizing their latent potential for user behavior modeling. To address these challenges, we propose the Multiplex Multimodal Representation Model (MMRM), a unified framework that aligns MLLMs with diverse collaborative signals. By employing a shared backbone with task-specific tokens and projection layers, MMRM simultaneously learns from multiple signals and generates comprehensive multiplex item representations in a single inference pass. Furthermore, we introduce a multiplex user representation strategy in ranking models, which derives task-specific user representations via search-based behavior sequence modeling leveraging multiplex item representations. Extensive experiments demonstrate MMRM’s superior efficiency and effectiveness. Notably, MMRM has been successfully deployed in the JD e-commerce search engine, yielding significant performance gains for millions of daily users.  \nCCS Concepts  \n• Information systems → Information retrieval.  \nThis work is licensed under a Creative Commons Attribution 4 .0 International License. SIGIR ’26, Melbourne, VIC, Australia  \n© 2026 Copyright held by the owner/author(s) .  \nACM ISBN 979-8-4007-2599-9/2026/07  \n[https://doi.org/10.1145/3805712.3808434](https://doi.org/10.1145/3805712.3808434)  \nKeywords  \nMultimodal Representation, Contrastive Learning, Multitask Learning, E-commerce Search System  \nACM Reference Format:  \nZhen-Lin Chen, Maosen Sheng, Peng Lin, Jianmin Chen, Zhuojian Xiao, Dongyue Wang, and Xiwei Zhao. 2026. MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia. ACM, New York, NY, USA, 6 pages. [https://doi.org/10.1145/](https://doi.org/10.1145/)[ ](https://doi.org/10.1145/)3805712.3808434  \n1 Introduction  \nE-commerce search engines rely on ranking models to capture finegrained user preferences across multiple objectives, such as ClickThrough Rate (CTR), Add-to-Cart Rate (ACR), and Conversion Rate (CVR) . Conventional ID-based ranking models inherently suffer from data sparsity and an inability to perceive product semantics (e.g., titles and images)[17, 25] . To address this, recent works finetune general Multimodal Large Language Models (MLLMs) using collaborative signals from e-commerce, then integrate the resulting representations into ranking models as auxiliary features[10, 11] .  \nDespite these advancements, two limitations persist. First, existing methods rely on a single collaborative signal for MLLM finetuning, such as search-based query-to-item (q2i) o","cbCairKrH3GpjHhw","https://ap.wps.com/l/cbCairKrH3GpjHhw","pdf",1077498,1,6,"English","en",105,"# Introduction\n## Limitations of prior multimodal ranking methods\n## Heterogeneous collaborative signals and multiplex representations\n# Proposed MMRM framework","[{\"question\":\"What are the two main limitations of existing multimodal ranking methods discussed in the abstract?\",\"answer\":\"Existing methods use only a single collaborative signal to fine-tune multimodal LLMs, missing heterogeneous signals needed for multitask ranking. They also treat multimodal representations as regular item features, failing to fully exploit them for modeling user behavior.\"},{\"question\":\"How does MMRM align MLLMs with multiple collaborative signals?\",\"answer\":\"MMRM uses a shared backbone with task-specific tokens and projection layers to simultaneously learn from multiple signals, producing multiplex item representations in a single inference pass.\"},{\"question\":\"How are user representations incorporated into the ranking model in MMRM?\",\"answer\":\"MMRM introduces a multiplex user representation strategy that derives task-specific user representations from search-based behavior sequence modeling, using the learned multiplex item representations.\"}]",1784202229,15,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"mmrm-a-multiplex-multimodal-representation-model-for-product-ranking-in-e-commerce-search","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/mmrm-a-multiplex-multimodal-representation-model-for-product-ranking-in-e-commerce-search/85278/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are the two main limitations of existing multimodal ranking methods discussed in the abstract?","Question",{"text":75,"@type":76},"Existing methods use only a single collaborative signal to fine-tune multimodal LLMs, missing heterogeneous signals needed for multitask ranking. They also treat multimodal representations as regular item features, failing to fully exploit them for modeling user behavior.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does MMRM align MLLMs with multiple collaborative signals?",{"text":80,"@type":76},"MMRM uses a shared backbone with task-specific tokens and projection layers to simultaneously learn from multiple signals, producing multiplex item representations in a single inference pass.",{"name":82,"@type":73,"acceptedAnswer":83},"How are user representations incorporated into the ranking model in MMRM?",{"text":84,"@type":76},"MMRM introduces a multiplex user representation strategy that derives task-specific user representations from search-based behavior sequence modeling, using the learned multiplex item representations.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]