[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85804-en":3,"doc-seo-85804-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85804,8796095462418,"Noah","https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780",8,"Research & Report","Conservation Laws for Diffusion Models","Diffusion models are typically trained with denoising objectives rather than exact likelihood, unlike autoregressive models. This work derives conservation laws from generalized extrinsic information transfer (GEXIT) functions for a broad class of memoryless noise processes. The data–model cross-entropy is characterized exactly as an integral of local information-theoretic derivatives along the noise path, unifying discrete and continuous diffusion. The Gaussian case recovers the I-MMSE relationship and yields a locality property via marginal posteriors, informing training and explaining performance differences from finite-capacity denoisers.","arXiv :2607 . 10067v 1 [ cs .LG] 11 Jul 2026  \nConservation Laws for Diffusion Models  \nZiv Aharoni, Henry D. Pfister  \nDepartment of Electrical and Computer Engineering  \nDuke University  \nDurham, NC, USA  \n{ziv.aharoni,[henry.pfister}@duke.edu](henry.pfister}@duke.edu)  \nAbstract  \nWhile autoregressive models optimize the exact data likelihood via the chain rule, diffusion models are typically trained with denoising objectives. We develop conservation laws based on generalized extrinsic information transfer (GEXIT) functions for a broad class of memoryless noise processes, showing that the data–model cross-entropy (CE) can be characterized exactly as an integral of local informationtheoretic derivatives along the noise path. This yields a unified characterization of the likelihood for discrete and continuous diffusion, with the Gaussian case reducing to the well-known mutual information–minimum mean-square error (I-MMSE) relationship. An immediate implication is a locality property: one can compute the information-theoretic derivatives using only the marginal posteriors along the noise path. As a result, training reduces to learning the marginal posteriors by minimizing the negative log-likelihood. While the conservation law implies that the entropy does not depend on the noise path, finite-capacity denoisers approximate the posteriors with varying accuracy across noise types, leading to differences in performance. We validate these predictions on synthetic Markov sources and standard benchmarks, including text8 and CIFAR-10. Code is available at [https:](https:)//[github.com/zivaharoni/conservation-laws-diffusion-models](github.com/zivaharoni/conservation-laws-diffusion-models).  \n1 Introduction  \nAutoregressive (AR) models treat the data distribution PX , for X = (X1 ,..., Xn), by factorizing it into a product of conditionals PX = Q PXi |X\u003Ci . Thus, minimizing the negative log-likelihood (NLL) of an AR model PθX = Q PθXi |X\u003Ci is equivalent to minimizing the cross-entropy between PX and PθX . This enables exact likelihood evaluation but requires sequential generation, which can amplify distribution-shift errors during long rollouts [1, 2] . Diffusion models offer a complementary factorization through a forward noising process where Y (τ) = (Y1 (τ),..., Yn (τ)) is a noisy version of X defined by PY (τ)|X . The parameter τ ∈ [0 , 1] orders the channels by degradation, with Y (0) corresponding to the original data and Y (1) to pure noise. The sampling (or reverse) process uses learned denoisers denoted PθX|Y(τ) .  \nThis step away from AR structure makes diffusion especially successful for image generation and increasingly attractive for text generation, including discrete diffusion, masked diffusion language models, and embedding-space text diffusion models [3–9] . Diffusion training typically optimizes local denoising objectives rather than exact sequence likelihood. Continuous denoising diffusion probabilistic model (DDPM)-style models are commonly motivated via variational bounds and equivalent weighted denoising losses [3, 4], while in discrete denoising diffusion probabilistic models (D3PMs) and related discrete settings the relationship between denoising objectives and likelihood is usually indirect [5–7] .  \nRecent work has begun to make this connection explicit by interpreting diffusion through informationtheoretic derivatives along the noising path, including Gaussian results based on I-MMSE curves  \nand discrete results for masked diffusion [10–12] . These analyses provide exact likelihood-scale characterizations for masked and Gaussian channels, but they do not yet yield a single formulation that applies across the broader class of memoryless noising mechanisms used in practice. This motivates the following question.  \nCan a single conservation law characterize the induced likelihood for diffusion models across memoryless noising processes, covering discrete and continuous diffusion?  \nOur starting point for answeri","cbCaioQb1bWR0bB6","https://ap.wps.com/l/cbCaioQb1bWR0bB6","pdf",6722536,1,23,"English","en",105,"# Abstract\n# Introduction\n## Autoregressive vs diffusion factorization\n## Entropy telescoping and conservation-law viewpoint\n## EXIT/GEXIT and local derivative structure","[{\"question\":\"What conservation law does the paper derive for diffusion models?\",\"answer\":\"It derives conservation laws expressing the data–model cross-entropy as an integral of local information-theoretic derivatives along the noise path, using generalized EXIT (GEXIT) functions for memoryless noise processes.\"},{\"question\":\"How do the results unify discrete and continuous diffusion?\",\"answer\":\"The formulation applies across a broad class of memoryless noise mechanisms, yielding a unified characterization of the induced likelihood for both discrete and continuous diffusion models.\"},{\"question\":\"What does the locality property mean for training diffusion models?\",\"answer\":\"Information-theoretic derivatives can be computed using only marginal posteriors along the noise path, so training reduces to learning these marginal posteriors by minimizing the negative log-likelihood.\"}]",1784206365,58,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"conservation-laws-for-diffusion-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/conservation-laws-for-diffusion-models/85804/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What conservation law does the paper derive for diffusion models?","Question",{"text":75,"@type":76},"It derives conservation laws expressing the data–model cross-entropy as an integral of local information-theoretic derivatives along the noise path, using generalized EXIT (GEXIT) functions for memoryless noise processes.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the results unify discrete and continuous diffusion?",{"text":80,"@type":76},"The formulation applies across a broad class of memoryless noise mechanisms, yielding a unified characterization of the induced likelihood for both discrete and continuous diffusion models.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the locality property mean for training diffusion models?",{"text":84,"@type":76},"Information-theoretic derivatives can be computed using only marginal posteriors along the noise path, so training reduces to learning these marginal posteriors by minimizing the negative log-likelihood.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]