[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127769-en":3,"doc-seo-127769-105":30,"detail-sidebar-cat-0-en-105":84},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},127769,1099523882182,"Eliana","https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8",8,"Research & Report","Input Perturbation Reduces Exposure Bias in Diffusion Models","Denoising Diffusion Probabilistic Models deliver high-quality samples, yet long sampling chains create high computational cost and an additional error accumulation effect along inference. The work identifies a training–testing discrepancy: training conditions on ground-truth noisy states, while testing conditions on previously generated states, causing drift analogous to exposure bias in autoregressive sequence generation. A simple training regularization is introduced by perturbing ground-truth samples to mimic inference-time prediction errors. Experiments show improved sample quality with unchanged recall and precision, plus faster training and inference, including a new CelebA 64×64 FID of 1.27 with 37.5% less training time.","This is the peer reviewd version of the followng article:  \nInput Perturbation Reduces Exposure Bias in Diffusion Models / Ning, M.; Sangineto, E.; Porrello, A.; Calderara, S.; Cucchiara, R. . -202:(2023), pp. 26245-26265. (Intervento presentato al convegno 40th International Conference on Machine Learning, ICML 2023 tenutosi a usa nel 2023) .  \nML Research Press Terms of use:  \nThe terms and conditions for the reuse of this version of the manuscript are specified in the publishing policy. For all terms of use and more information see the publisher's website.  \n01/09/2024 14:42  \n(Article begins on next page)  \nInput Perturbation Reduces Exposure Bias in Diffusion Models  \nMang Ning 1 2 Enver Sangineto 2 Angelo Porrello 2 Simone Calderara 2 Rita Cucchiara 2  \narXiv :2301 . 11706v3 [ cs .LG] 18 Jun 2023  \nAbstract  \nDenoising Diffusion Probabilistic Models have shown an impressive generation quality although their long sampling chain leads to high computational costs. In this paper, we observe that along sampling chain also leads to an error accumulation phenomenon, which is similar to the exposure bias problem in autoregressive text generation. Specifically, we note that there is a discrepancy between training and testing, since the former is conditioned on the ground truth samples, while the latter is conditioned on the previously generated results. To alleviate this problem, we propose a very simple but effective training regularization, consisting in perturbing the ground truth samples to simulate the inference time prediction errors. We empirically show that, without affecting the recall and precision, the proposed input perturbation leads to a significant improvement in the sample quality while reducing both the training and the inference times. For instance, on CelebA 64×64, we achieve a new state-of-theart FID score of 1.27, while saving 37.5% of the training time. The code is available at [https:](https:)//[github.com/forever208/DDPM-IP](github.com/forever208/DDPM-IP).  \n1. Introduction  \nDenoising Diffusion Probabilistic Models (DDPMs) (SohlDickstein et al., 2015 ; Ho et al., 2020) are a new generative paradigm which is attracting a growing interest due to its very high-quality sample generation capabilities (Dhariwal & Nichol, 2021 ; Nichol et al., 2022 ; Ramesh et al., 2022) . Differently from most existing generative methods which synthesize a new sample in a single step, DDPMs resemble the Langevin dynamics (Welling & Teh, 2011) and the generation process is based on a sequence of denoising steps, in which a synthetic sample is created starting from pure noise  \n1Department of Information and Computing Science, Utrecht University, the Netherlands. 2Department of Engineering (DIEF), University of Modena and Reggio Emilia, Italy. Correspondence to: Mang Ning \u003C[m.ning@uu.nl](m.ning@uu.nl) >, Enver Sangineto \u003Cen[ver.sangineto@unimore.it](ver.sangineto@unimore.it) >.  \nPublished as a conference paper at ICML 2023 .  \nand autoregressively reducing the noise component. In more detail, during training, a real sample x0 is progressively destroyed in T steps adding Gaussian noise (forward process) . The sequence x0 , ... , xt , ... , xT so obtained, is used to train a deep denoising autoencoder (µ(·)) to invert the forward process: xˆxxt−1 = µ (xt , t) . At inference time, the generation process is autoregressive because it depends on the previously generated samples: xˆxxt−1 = µ (xˆxxt , t) (Sec. 3) .  \nDespite the large success of DDPMs in different generative fields (Sec. 2), one of the main drawbacks of these models is their very long computational time, which depends on the large number of steps T required at both the training and the inference stage. As recently emphasised in (Xiao et al., 2022), the fundamental reason why T needs to be large is that each denoising step is assumed to be Gaussian, and this assumption holds only for small step sizes. Conversely, with larger step sizes, the prediction network (µ(·)) need","cbCaipUhE4tJueCc","https://ap.wps.com/l/cbCaipUhE4tJueCc","pdf",9912091,1,22,"English","en",105,"# Introduction\n## Background on DDPMs\n## Training–Inference Discrepancy and Exposure Bias\n## Proposed Input Perturbation Regularization\n## Experimental Results and Efficiency Gains","[{\"question\":\"What improvements does the paper report experimentally?\",\"answer\":\"The approach significantly improves sample quality without affecting recall and precision, and it reduces both training and inference times, e.g., achieving FID 1.27 on CelebA 64×64 with 37.5% less training time.\"}]","Input Perturbation Reduces Exposure Bias in Diffusion Models | PDF",1785941514,55,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":79,"head_meta":81,"extra_data":83,"updated_unix":28},"input-perturbation-reduces-exposure-bias-in-diffusion-models","",{"@graph":36,"@context":78},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/input-perturbation-reduces-exposure-bias-in-diffusion-models/127769/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-22","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72],{"name":73,"@type":74,"acceptedAnswer":75},"What improvements does the paper report experimentally?","Question",{"text":76,"@type":77},"The approach significantly improves sample quality without affecting recall and precision, and it reduces both training and inference times, e.g., achieving FID 1.27 on CelebA 64×64 with 37.5% less training time.","Answer","https://schema.org",{"og:url":52,"og:type":80,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":82,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":85},[86,90,94,98,103,108,113,116,121,124,128],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":87,"show_sort_weight":88,"slug":89},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":91,"show_sort_weight":92,"slug":93},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Exam",70,"exam",{"id":99,"doc_module":4,"doc_module_name":46,"category_name":100,"show_sort_weight":101,"slug":102},5,"Comic",60,"comic",{"id":104,"doc_module":4,"doc_module_name":46,"category_name":105,"show_sort_weight":106,"slug":107},6,"Technology",50,"technology",{"id":109,"doc_module":4,"doc_module_name":46,"category_name":110,"show_sort_weight":111,"slug":112},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":114,"slug":115},30,"research-report",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},9,"Religion & Spirituality",20,"religion-spirituality",{"id":119,"doc_module":4,"doc_module_name":46,"category_name":122,"show_sort_weight":119,"slug":123},"World Cup","world-cup",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":125,"slug":127},10,"Lifestyle","lifestyle",{"id":129,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":99,"slug":131},19,"General","general"]