[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-289798-105":59,"doc-detail-289798-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","speech-separation-with-pretrained-frontend-to-minimize-domain-mismatch","Speech Separation with Pretrained Frontend to Minimize Domain Mismatch","","Speech separation aims to disentangle individual utterances from multi-speaker mixtures. Because real-world cocktail-party recordings often lack clean reference targets, most separation systems are trained on synthetic data, creating a domain gap in deployment. The study presents a self-supervised domain-invariant pretrained (DIP) frontend that learns from unlabeled mixtures without target speech. A Siamese architecture with mixture predictive coding (MPC) and mixture invariant coding (MIC) provides transferable contextual cues; the frontend is then frozen during downstream training using synthetic data. Feature-resolution alignment and evaluations on benchmarks and real datasets validate DIP’s superiority and improved intelligibility.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/speech-separation-with-pretrained-frontend-to-minimize-domain-mismatch/289798/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/speech-separation-with-pretrained-frontend-to-minimize-domain-mismatch/289798.png","ImageObject",300,407,{"name":92,"@type":93},"Aditya","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-29","2026-09-17",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",16,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why do speech separation models face performance drops in real-world use?","Question",{"text":113,"@type":114},"Most models are trained on synthetic anechoic data, which differs from real recordings with unpredictable attenuation and reverberation, leading to a domain mismatch.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"What is the proposed DIP frontend and how is it trained?",{"text":118,"@type":114},"The DIP frontend is self-supervised and domain-invariant, trained on unlabeled mixture data without target reference speech using a Siamese network with mixture predictive coding (MPC) and mixture invariant coding (MIC).",{"name":120,"@type":111,"acceptedAnswer":121},"How does the method connect pretraining to downstream separation models?",{"text":122,"@type":114},"The DIP frontend is frozen as a feature extractor during training of downstream speech separation models on synthetic data, and a separation pipeline aligns the feature resolution for effective transfer.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},289798,1789637246,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},962085564549,"https://ap-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45","Speech Separation with Pretrained Frontend to Minimize Domain Mismatch  \nWupeng Wang,Zexu Pan,,Member, IEEE, Xinke Li,Shuai Wang,,Member, IEEE,and Haizhou Li,,Fellow, IEEE  \narXiv :2411 .03085v 1 [ cs . SD] 5 Nov 2024  \nAbstract—Speech separation seeks to separate individual speech signals from a speech mixture. Typically, most separation models are trained on synthetic data due to the unavailability of target reference in real-world cocktail party scenarios. As a result, there exists a domain gap between real and synthetic data when deploying speech separation models in real-world applications. In this paper, we propose a self-supervised domaininvariant pretrained (DIP) frontend that is exposed to mixture data without the need for target reference speech. The DIP frontend utilizes a Siamese network with two innovative pretext tasks, mixture predictive coding (MPC) and mixture invariant coding (MIC), to capture shared contextual cues between real and synthetic unlabeled mixtures. Subsequently, we freeze the DIP frontend as a feature extractor when training the downstream speech separation models on synthetic data. By pretraining the DIP frontend with the contextual cues, we expect that the speech separation skills learned from synthetic data can be effectively transferred to real data. To benefit from the DIP frontend, we introduce a novel separation pipeline to align the feature resolution of the separation models. We evaluate the speech separation quality on standard benchmarks and realworld datasets. The results confirm the superiority of our DIP frontend over existing speech separation models. This study underscores the potential of large-scale pretraining to enhance the quality and intelligibility of speech separation in real-world applications.  \nIndex Terms—Speech Separation, Self-supervise, Pretraining, Siamese Network, Maximum Mean Discrepancy  \nI. INTRODUCTION  \nSPEECH separation seeks to separate individual speech sig  \nnals in a complex, multi-speaker acoustic environments, commonly known as the ‘cocktail party problem’ [1] . Speech separation has been an active research, not only because it is a human cognitive ability, but also it serves as a frontend in many practical speech processing tasks, such as speaker recognition [2]–[4], automatic speech recognition [5]–[7], and voice conversion [8], [9] .  \nThe research is supported by National Natural Science Foundation of China (Grant No. 62271432), Shenzhen Science and Technology Program ZDSYS20230626091302006, and Shenzhen Science and Technology Research Fund (Fundamental Research Key Project Grant No. JCYJ20220818103001002) . (Corresponding author: Shuai Wang) .  \nWupeng Wang is with the Department of Electrical and Computer Engineering, National University of Singapore, 119077 Singapore (e-mail: [e0125301@u.nus.edu](e0125301@u.nus.edu)) .  \nZexu Pan is with the Alibaba Group, 189554 Singapore (e-mail: [zexu.pan@alibaba-inc.com](zexu.pan@alibaba-inc.com)).  \nXinke Li is with the Industrial Systems Engineering and Management, National University of Singapore, 119077 Singapore (e-mail: [xinke.li@u.nus.edu](xinke.li@u.nus.edu)) .  \nShuai Wang and Haizhou Li are with the Shenzhen Research Institute of Big Data, Chinese University of Hong Kong, Shenzhen, Guangdong, 518172 P.R. China ([e-mail: wangshuai@cuhk.edu.cn](e-mail: wangshuai@cuhk.edu.cn); [haizhouli@cuhk.edu.cn](haizhouli@cuhk.edu.cn)).  \nTraditional speech separation technique includes independent component analysis (ICA) [10]–[12], non-negative matrix factorization (NMF) [13]–[15] or the theory of computational auditory scene analysis (CASA) [16]–[18] . With the advent of deep learning, supervised speech separation models, including both time-domain [19]–[21] and frequency-domain models [22]–[24], have achieved remarkable advancements. Despite impressive results, these models don’t generalize well to real-world scenarios. This is primarily due to the fact that most speech separation models are trained on synth","cbCaidszkYWYtuNb","https://ap.wps.com/l/cbCaidszkYWYtuNb","pdf",709904,15,"English","# Abstract\n# Index Terms\n# I. Introduction","[{\"question\":\"Why do speech separation models face performance drops in real-world use?\",\"answer\":\"Most models are trained on synthetic anechoic data, which differs from real recordings with unpredictable attenuation and reverberation, leading to a domain mismatch.\"},{\"question\":\"What is the proposed DIP frontend and how is it trained?\",\"answer\":\"The DIP frontend is self-supervised and domain-invariant, trained on unlabeled mixture data without target reference speech using a Siamese network with mixture predictive coding (MPC) and mixture invariant coding (MIC).\"},{\"question\":\"How does the method connect pretraining to downstream separation models?\",\"answer\":\"The DIP frontend is frozen as a feature extractor during training of downstream speech separation models on synthetic data, and a separation pipeline aligns the feature resolution for effective transfer.\"}]","Speech Separation with Pretrained Frontend to Minimize Domain Mismatch | PDF",38]