[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125060-en":3,"doc-seo-125060-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},125060,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Sanitized Clustering Against confounding Bias - SCAB","Real-world datasets contain biases introduced by different collection conditions, and these inconsistencies act as confounding factors that distort unsupervised cluster analysis. Prior approaches often remove such bias by projecting data onto the orthogonal complement of a confounding subspace, relying on linear correlations between data and confounders, which limits performance on complex, non-linearly related data. This work proposes Sanitized Clustering Against confounding Bias (SCAB), which removes confounding information in a semantic latent space using a non-linear dependence measure via variational auto-encoder and mutual-information minimization, then clusters purified latent representations with a dedicated module. Experiments on complex datasets demonstrate significant clustering gains.","Sanitized clustering against confounding bias  \nYinghua Yao1,2,3,4 · Yuangang Pan1,2 · Jing Li1,2 · Ivor W. Tsang1,2,4 · Xin Yao3  \nReceived: 1 June 2023 / Revised: 10 August 2023 / Accepted: 7 October 2023 © The Author(s) 2023  \nAbstract  \nReal-world datasets inevitably contain biases that arise from different sources or conditions during data collection. Consequently, such inconsistency itself acts as a confounding factor that disturbs the cluster analysis. Existing methods eliminate the biases by projecting data onto the orthogonal complement of the subspace expanded by the confounding factor before clustering. Therein, the interested clustering factor and the confounding factor are coarsely considered in the raw feature space, where the correlation between the data and the confounding factor is ideally assumed to be linear for convenient solutions. These approaches are thus limited in scope as the data in real applications is usually complex and non-linearly correlated with the confounding factor. This paper presents a new clustering framework named Sanitized Clustering Against confounding Bias, which removes the confounding factor in the semantic latent space of complex data through a non-linear dependence measure. To be specific, we eliminate the bias information in the latent space by minimizing the mutual information between the confounding factor and the latent representation delivered by variational auto-encoder. Meanwhile, a clustering module is introduced to cluster over the purified latent representations. Extensive experiments on complex datasets demonstrate that our SCAB achieves a significant gain in clustering performance by removing the confounding bias.  \nKeywords Deep clustering · Confounding bias · Mutual information · Non-linear dependence  \n1 Introduction  \nClustering is an essential technique for unsupervised data analysis, whose objective is to partition samples into groups so that the samples in the same group are similar while those from different groups are significantly different (Jain et al., 1999) . Standard clustering methods (Cheng, 1995 ; Modha & Spangler, 2003 ; Xie et al., 2016) is capable of capturing the desired semantic structure embedded in the clean raw data. However, biases are  \nEditors: Vu Nguyen, Dani Yogatama.  \nThe first version of this work was done when the first author was at SUSTech.  \nExtended author information available on the last page of the article  \n1 3  \nFig. 1 The architecture of our sanitized clustering against confounding Bias (SCAB)  \ninherently present in real-world datasets, as they emerge from data collected across diverse times, scenarios, or platforms (Listgarten et al., 2010 ; Jacob et al., 2016 ; Li et al., 2020) . These biases may introduce confounding factors that bring spurious correlation (Wu et al., 2023), obscuring the true underlying clustering structure (Listgarten et al., 2010), named as confounding biases in this paper. Despite the inevitable presence of data biases, we argue that the bias information can be identified by domain experts (Chierichetti et al., 2017 ; Benito et al., 2004) and easily accessible (e.g., the data source usually denoted in metadata) . In this study, we perform clustering while removing the negative effect of the bias.  \nPrevious methods (Jacob et al., 2016 ; Gagnon-Bartsch & Speed, 2012) simply project raw data onto the subspace orthogonal to the space expanded by the confounding factor under the linear assumption before clustering. Specifically, they decompose the data into linear combinations of the desired clustering factor and the confounding factor. In the linear space, they remove the bias information by simply subtracting the confounding covariate from the data. In parallel, Benito et al. (2004) applied an improved SVM which finds alinear hyperplane to separate two classes (i.e., the binary confounding factor that indicates the data source) in a supervised manner and then projects the raw data on this hyperplane. Su","cbCaidh418XNahZC","https://ap.wps.com/l/cbCaidh418XNahZC","pdf",2184659,1,20,"English","en",105,"# Abstract\n# Introduction\n## Problem and motivation: confounding bias in clustering\n## Limitations of existing projection-based methods\n## Proposed framework: Sanitized Clustering Against confounding Bias (SCAB)","[{\"question\":\"Why do confounding biases hinder clustering in real-world data?\",\"answer\":\"Biases arising from different collection times, scenarios, or platforms create confounding factors and spurious correlations, obscuring the true underlying clustering structure.\"},{\"question\":\"How does SCAB remove confounding bias?\",\"answer\":\"SCAB minimizes mutual information between the confounding factor and the latent representation produced by a variational auto-encoder, yielding a latent space purified of bias-related information.\"},{\"question\":\"What allows SCAB to work better on complex data than earlier methods?\",\"answer\":\"SCAB uses deep representation learning and a non-linear dependence measure, avoiding the overly restrictive linear assumptions and enabling semantic clustering directly in the latent space.\"}]","Sanitized Clustering Against confounding Bias - SCAB | PDF",1785896410,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"sanitized-clustering-against-confounding-bias-scab","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/sanitized-clustering-against-confounding-bias-scab/125060/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do confounding biases hinder clustering in real-world data?","Question",{"text":75,"@type":76},"Biases arising from different collection times, scenarios, or platforms create confounding factors and spurious correlations, obscuring the true underlying clustering structure.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SCAB remove confounding bias?",{"text":80,"@type":76},"SCAB minimizes mutual information between the confounding factor and the latent representation produced by a variational auto-encoder, yielding a latent space purified of bias-related information.",{"name":82,"@type":73,"acceptedAnswer":83},"What allows SCAB to work better on complex data than earlier methods?",{"text":84,"@type":76},"SCAB uses deep representation learning and a non-linear dependence measure, avoiding the overly restrictive linear assumptions and enabling semantic clustering directly in the latent space.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]