[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-327717-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-327717-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","efficient-bias-mitigation-in-t2i-diffusion-models-using-concept-graphs","Efficient bias mitigation in T2I diffusion models using - Concept Graphs","","Text-to-Image diffusion models often propagate harmful bias inherited from training data, and many existing mitigation methods intervene only at the text encoder or apply inference-time guidance, which can produce semantically incoherent generations. CO-ALIGN (Concept Ontology Alignment) addresses these limitations by aligning concept graphs within the model’s internal concept ontology. It jointly aligns concepts in the text encoder and denoiser to reduce bias while preserving generative integrity. Experiments across multiple paradigms show substantial fairness gains and large reductions in incoherent outputs, plus improved unlearning robustness in downstream tasks.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/efficient-bias-mitigation-in-t2i-diffusion-models-using-concept-graphs/327717/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/efficient-bias-mitigation-in-t2i-diffusion-models-using-concept-graphs/327717.png","ImageObject",300,407,{"name":42,"@type":43},"Oliver","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-31","2026-09-21",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does CO-ALIGN target in text-to-image diffusion models?","Question",{"text":63,"@type":64},"CO-ALIGN targets harmful demographic bias that is inherited from training data and is reproduced through learned concept associations inside both the text encoder and the denoiser.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"Why do existing bias mitigation methods often produce semantically incoherent outputs?",{"text":68,"@type":64},"They typically correct only one component (either the text encoder or the denoiser). Without disentanglement guarantees, the other component receives out-of-distribution conditioning and cannot coherently interpret it, leading to incoherent generations.",{"name":70,"@type":61,"acceptedAnswer":71},"How does CO-ALIGN mitigate bias while preserving generative coherence?",{"text":72,"@type":64},"CO-ALIGN jointly aligns concept graphs by editing the structured relational graph of concepts across the text encoder embedding space and the denoiser cross-attention layers toward a target topology, restoring both coherence and balanced representation.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},327717,1790378789,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":127,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":143,"read_time":144,"preview_page_count":4},8796095461610,"https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c","Efficient bias mitigation in T2I diffusion models using  \nConcept Graphs  \nMansi  \nDepartment of Computing Imperial College London [m.-24@imperial.ac.uk](m.-24@imperial.ac.uk)  \nAvinash Kori  \nDepartment of Computing Imperial College London [a.kori21@imperial.ac.uk](a.kori21@imperial.ac.uk)  \nFrancesco Leofante  \nDepartment of Computing Imperial College London [f.leofante@imperial.ac.uk](f.leofante@imperial.ac.uk)  \narXiv :2607 .03397v 1 [ cs .AI] 3 Jul 2026  \nAbstract  \nText-to-Image diffusion models often propagate harmful bias inherited from the training data. Existing bias mitigation techniques typically intervene only at the text encoder or provide inference-time guidance, often leading to generations that collapse into semantically incoherent outputs. To address these limitations, we introduce CO-ALIGN (Concept Ontology Alignment), a novel bias mitigation approach based on concept-graph alignment that operates on the model’s internal concept ontology. By aligning concepts within the text encoder and denoiser, CO-ALIGN achieves substantial bias reduction while preserving generative integrity. We demonstrate the effectiveness of concept-graph alignment across three paradigms: text-encoders, denoisers and joint text-denoiser ontology alignment. CO-ALIGN outperforms the state of the art, improving fairness by 30%, ∆FID = 11 .4 in image quality, 2 .8% in image fidelity, all while reducing semantically incoherent outputs by 88% . Beyond bias mitigation, we show that CO-ALIGN benefits other downstream tasks as well. In particular, our experiments demonstrate that better-aligned internal ontologies enhance concept unlearning robustness across multiple unlearning techniques.  \n1 Introduction  \nText-to-image (T2I) diffusion models have achieved remarkable generative capability, yet they systematically reproduce and amplify the demographic biases present in their training corpora [Schuhmann et al., 2022a, Bianchi et al., 2023] . These biases are not just superficial stylistic artifacts, but structural properties of the model’s learned concept associations. Once embedded in the weights, they are reproduced consistently at every generation. To address this, current bias mitigation methods intervene at one of two levels: the text encoder, via text embedding fine-tuning [Shen et al., 2024] or inferencetime steering [Friedrich et al., 2023, Chuang et al., 2023]; or the denoiser, via cross-attention weight editing [Orgad et al., 2023, Gandikota et al., 2024] or latent space manipulation [Parihar et al., 2024, Li et al., 2024, Shi et al., 2025] . Both paradigms share a common and consequential assumption: that the text encoder and denoiser can be treated as independent correction targets, and that fixing one is sufficient to correct the generated output.  \nThe problem with existing approaches. The denoiser usually adapts to the given text-encoder, orin the case of end-to-end training, both text encoder and the denoiser are jointly updated, without explicit disentanglement guarantees: the denoiser’s cross-attention heads learn to expect the specific biased embedding geometry produced by the original text encoder, and have organized their internal concept representations around producing biased visual outputs. When only the text encoder is edited, the denoiser receives an out-of-distribution conditioning signal, which it often cannot interpret coherently, resulting in semantically incoherent generations that depict neither the target concept nor any meaningful content. Conversely, when only the denoiser is edited, the text encoder continues to  \nPreprint.  \nFigure 1: Existing bias mitigation methods that intervene on only the text encoder (left) or denoiser (centre) produce either semantically incoherent outputs or gender-biased generations for the prompt“Nurse”. CO-ALIGN (right) jointly aligns the concept graphs of both components toward a target topology, restoring both coherence and demographic balance. CG representes the concept graphs.","cbCaijMQlIWAhD8y","https://ap.wps.com/l/cbCaijMQlIWAhD8y","pdf",3123570,"English","# Abstract\n# 1 Introduction\n## The problem with existing approaches\n## Our approach\n## Emergent property in CO-ALIGN","[{\"question\":\"What problem does CO-ALIGN target in text-to-image diffusion models?\",\"answer\":\"CO-ALIGN targets harmful demographic bias that is inherited from training data and is reproduced through learned concept associations inside both the text encoder and the denoiser.\"},{\"question\":\"Why do existing bias mitigation methods often produce semantically incoherent outputs?\",\"answer\":\"They typically correct only one component (either the text encoder or the denoiser). Without disentanglement guarantees, the other component receives out-of-distribution conditioning and cannot coherently interpret it, leading to incoherent generations.\"},{\"question\":\"How does CO-ALIGN mitigate bias while preserving generative coherence?\",\"answer\":\"CO-ALIGN jointly aligns concept graphs by editing the structured relational graph of concepts across the text encoder embedding space and the denoiser cross-attention layers toward a target topology, restoring both coherence and balanced representation.\"}]","Efficient bias mitigation in T2I diffusion models using - Concept Graphs | PDF",1789977902,48]