[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86420-en":3,"doc-seo-86420-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},86420,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","CMSL: Constructive Multi-Sequence Learning for Recommendation Systems","CMSL（Constructive Multi-Sequence Learning）提出一种面向推荐系统的序列学习新范式，针对现有方法将用户历史当作单一“prompt”导致的上下文污染问题。研究指出自然语言具备线性叙事统一性，而推荐交互本质上是多主题碎片，弱相关行为会争夺注意力并稀释高意图信号。为此，CMSL通过可学习的Sequence Construction Module在潜空间解耦历史为多条“纯”主题脉络，并用线性注意力高效建模，从而降低跨意图干扰并提升排序与检索能力。","arXiv :2606 .28533v2 [ cs .IR] 10 Jul 2026  \nCMSL: Constructive Multi-Sequence Learning for Recommendation Systems  \nZikun Cui 1 ,∗ , Renzhi Wu 1 ,∗ , Junjie Yang 1 ,∗ , Li Sheng 1 ,∗ , Jijie Wei 1 ,∗ , Linfeng Liu 1 ,∗ , Tai Guo 1 ,∗ , Tao Jia 1 ,∗ , Xiaodong Wang2 ,∗ , Hong Li 1 ,∗ , Li Yu3 ,∗ , Sri Reddy 1 ,∗ , Hong Yan 1 ,∗  \n1 Meta MRS, 2 Meta PyTorch, 3 Meta FB Monetization  \n∗ All authors contributed equally to this research  \nSequence learning has emerged as the promising paradigm in recommendation systems, surpassing traditional Deep Learning Recommendation Models (DLRM) by capturing the temporal nuances of user behavior. However, current state-of-the-art architectures operate under a limiting analogy: they treat user history as a monolithic chronological sequence like a sentence in a Large Language Model (LLM) . We observe a fundamental divergence between natural language and recommendation data: unlike the linear, logical flow of text, user history is inherently multi-faceted. A user’s journey is a fragmented reflection of diverse interests, resulting in much weaker coherence between items than is found in LLM training data. This lack of structural unity leads to context pollution. In single-sequence modeling, unrelated behaviors compete for the same attention budget. This “noisy”signal dilutes the model’s focus, effectively capping its ability to discern high-intent patterns from background activity. To address this, we propose Constructive Multi-Sequence Learning (CMSL), a paradigm shift from passive sequence ingestion to active “context engineering” that constructs multiple coherent sequences in latent space. CMSL leverages a learnable Sequence Construction Module to disentangle user history into “pure” thematic strands, followed by a linear attention mechanism to efficiently model these strands at scale. CMSL has been deployed across ranking and retrieval tasks and across four major surfaces at Meta.  \nDate: July 14, 2026  \nCorrespondence: [renzhiwu@meta.com](renzhiwu@meta.com)  \nKeywords: Recommendation systems, Sequence learning, Generative sequence modeling  \n1 Introduction  \nBuilding on the success of Transformers in language and vision tasks, sequence learning has emerged asa powerful paradigm for recommendation systems. Unlike the previous DLRM approaches (Covington et al. , 2016 ; Cheng et al. , 2016 ; Zhou et al. , 2018 ; Tang et al. , 2020 ; Wang et al. , 2021 ; Mudigere et al. , 2022 ; Xia et al. , 2023 ; Zhang et al. , 2024), which rely heavily on aggregated categorical features and often loses sequential information, sequence learning retains important insights into the user’s journey. A large body of work has developed increasingly powerful sequential recommenders, demonstrating notable success (Kang and McAuley, 2018 ; Zhai et al. , 2024 ; Zhang et al. , 2025 ; Zeng et al. , 2025 ; Deng et al. , 2025) .  \nA notable trend is that advances in sequential recommendation have increasingly been inspired by LLM research (Kang and McAuley, 2018 ; Zhai et al. , 2024 ;  \nZhang et al. , 2025 ; Zeng et al., 2025 ; Deng et al., 2025) . Many RecSys models adopt a similar interface: represent a user’s interaction history as a sequence of tokens (item IDs), feed it into a self-attention backbone, and treat next-item prediction as an autoregressive continuation problem. This “history-as-prompt” view is attractive because it provides a unified modeling recipe and a straightforward path to scaling with model size and context length.  \nThe Context Pollution Problem: While a sentence is governed by strict syntactic rules and linear semantic dependencies, a user’s digital footprint is a “noisy mosaic.” It is a disjointed mixture of overlapping interests, exploratory clicks, and accidental interactions. Because recommendation data lacks the inherent narrative unity found in text, direct applications of LLM-style sequence modeling often struggle to capture the underlying user intent effectively.  \nIrrelevant or weakly rel","cbCaijELwDD3e7uv","https://ap.wps.com/l/cbCaijELwDD3e7uv","pdf",986503,2,1,7,"English","en",105,"# Introduction\n## Sequence learning in recommendation\n## The context pollution problem\n## From modeling a sequence to constructing context\n## Proposed CMSL approach","[{\"question\":\"CMSL试图解决推荐系统中的什么核心问题？\",\"answer\":\"CMSL主要针对“上下文污染”问题：现有模型把用户历史当作单一时间序列或prompt，会让无关或弱相关行为争夺注意力，从而稀释真实意图并干扰表示学习。\"},{\"question\":\"CMSL与传统单序列建模相比做了哪些关键改变？\",\"answer\":\"CMSL不再被动摄入单一序列，而是主动在潜空间构造多条相互一致的主题子序列；每条子序列分别进行自注意力，从而减少跨意图干扰。\"},{\"question\":\"CMSL的主要组件如何工作？\",\"answer\":\"CMSL使用可学习的Sequence Construction Module将用户交互解耦为“纯”主题脉络，随后采用线性注意力机制对这些脉络进行高效建模，以支持大规模任务。\"}]",1784211633,18,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"cmsl-constructive-multi-sequence-learning-for-recommendation-systems","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/cmsl-constructive-multi-sequence-learning-for-recommendation-systems/86420/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"CMSL试图解决推荐系统中的什么核心问题？","Question",{"text":75,"@type":76},"CMSL主要针对“上下文污染”问题：现有模型把用户历史当作单一时间序列或prompt，会让无关或弱相关行为争夺注意力，从而稀释真实意图并干扰表示学习。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"CMSL与传统单序列建模相比做了哪些关键改变？",{"text":80,"@type":76},"CMSL不再被动摄入单一序列，而是主动在潜空间构造多条相互一致的主题子序列；每条子序列分别进行自注意力，从而减少跨意图干扰。",{"name":82,"@type":73,"acceptedAnswer":83},"CMSL的主要组件如何工作？",{"text":84,"@type":76},"CMSL使用可学习的Sequence Construction Module将用户交互解耦为“纯”主题脉络，随后采用线性注意力机制对这些脉络进行高效建模，以支持大规模任务。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]