[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85460-en":3,"doc-seo-85460-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85460,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","CANDI Hybrid Discrete-Continuous Diffusion Models","CANDI (Continuous And Discrete diffusion) addresses the gap between continuous diffusion success in continuous domains and its weaker performance on discrete data. The work introduces token identifiability to analyze how Gaussian noise corrupts discrete tokens through discrete identity corruption and continuous rank degradation. These effects scale differently with vocabulary size, creating a tradeoff between learning continuous geometry and discrete structure. CANDI decouples both corruption processes, improving discrete generation and enabling classifier guidance via gradient addition and stronger text generation at low NFE compared with masked diffusion.","CANDI: Hybrid Discrete-Continuous Diffusion Models  \nPatrick Pynadath 1 Jiaxin Shi 2 Ruqi Zhang 1  \narXiv :2510 .225 10v 3 [ cs .LG] 11 Jul 2026  \nAbstract  \nWhile continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete formulations.  \nTo understand this gap, we introduce token identifiability, an analytical framework characterizing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance that forces a tradeoff between learning continuous geometry and discrete structure. To address this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page:  \n[https://patrickpynadath1.github](https://patrickpynadath1.github) .  \nio/candi-lander.  \n1. Introduction  \nDiffusion models have become a central tool in generative modeling, with score-based methods showing remarkable performance across a range of continuous domains (SohlDickstein et al., 2015 ; Song & Ermon, 2019 ; Song et al., 2020a ; Karras et al., 2022) . Recent works have adapted diffusion to discrete settings by training with discrete noise instead of continuous Gaussian noise (Austin et al., 2023 ;  \n1Purdue University, West Lafayette, U.S.A 2 Google Deepmind. Correspondence to: Patrick Pynadath \u003C[ppynadat@purdue.edu](ppynadat@purdue.edu) >, Ruqi Zhang \u003C[ruqiz@purdue.edu](ruqiz@purdue.edu) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \nHoogeboom et al., 2021 ; Lou et al., 2024 ; Sun et al., 2023) . These pure discrete diffusion frameworks have achieved significant success, scaling to large language models and demonstrating strong performance across various text generation tasks (Shi et al., 2024 ; Ye et al., 2025 ; Nie et al., 2024) . Moreover, such methods have generally outperformed direct applications of continuous diffusion to discrete spaces (Arriola et al., 2025 ; Sahoo et al., 2024) .  \nThis outperformance is not obvious a priori. Continuous diffusion, in principle, should have advantages: it learns a score function that jointly updates multiple positions and captures correlations among variables (Song et al., 2020b ; Lou et al., 2024 ; Sahoo et al., 2024) . Such parallel position updates are valuable, especially in compute-constrained settings, where few function evaluations are allowed. Yet, empirical results show continuous approaches have underperformed pure discrete approaches even at low NFE, contradicting these expectations.  \nIn this paper, we investigate the cause of continuous diffusion’s underperformance on discrete data and propose a simple yet effective solution to recover its benefits. We introduce token identifiability as a framework for analyzing how continuous Gaussian noise interacts with discrete structure, characterizing signal corruption along two axes: discrete identity corruption, which measures if an incorrect token is closest to the noisy latent, and continuous rank degradation, which measures how many incorrect tokens are closer to the noisy latent than the correct token. Both are crucial, as discrete identity corruption directly relates to learning conditional dependencies, while continuous rank degradation enables the learning of a continuous score function through denoisin","cbCaidM6yMS4a1nE","https://ap.wps.com/l/cbCaidM6yMS4a1nE","pdf",1533135,1,33,"English","en",105,"# Introduction\n## Token identifiability and the mismatch\n## CANDI hybrid framework and decoupled corruption\n## Guidance and text generation results","[{\"question\":\"What problem does CANDI aim to solve in diffusion models for discrete data?\",\"answer\":\"Continuous diffusion models work well for continuous generation but underperform on discrete data. CANDI studies why this happens and proposes a hybrid method to recover continuous diffusion benefits in discrete spaces.\"},{\"question\":\"What is token identifiability in the paper?\",\"answer\":\"Token identifiability is an analytical framework describing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. The paper uses this to explain the scaling mismatch with vocabulary size.\"},{\"question\":\"How does CANDI enable learning both continuous geometry and discrete conditional structure?\",\"answer\":\"CANDI uses a hybrid corruption strategy that preserves selected positions via discrete masking while applying Gaussian noise to others. This decouples discrete corruption from Gaussian noise dynamics, letting both learning objectives scale reliably together.\"}]",1784203712,83,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"candi-hybrid-discrete-continuous-diffusion-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/candi-hybrid-discrete-continuous-diffusion-models/85460/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does CANDI aim to solve in diffusion models for discrete data?","Question",{"text":75,"@type":76},"Continuous diffusion models work well for continuous generation but underperform on discrete data. CANDI studies why this happens and proposes a hybrid method to recover continuous diffusion benefits in discrete spaces.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is token identifiability in the paper?",{"text":80,"@type":76},"Token identifiability is an analytical framework describing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. The paper uses this to explain the scaling mismatch with vocabulary size.",{"name":82,"@type":73,"acceptedAnswer":83},"How does CANDI enable learning both continuous geometry and discrete conditional structure?",{"text":84,"@type":76},"CANDI uses a hybrid corruption strategy that preserves selected positions via discrete masking while applying Gaussian noise to others. This decouples discrete corruption from Gaussian noise dynamics, letting both learning objectives scale reliably together.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]