[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-290240-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-290240-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","progressive-tempering-sampler-with-diffusion","Progressive Tempering Sampler with Diffusion","","Recent research designs neural samplers to amortize sampling from unnormalized densities, yet they remain less efficient than Parallel Tempering (PT) MCMC in terms of target evaluation. PT can yield only dependent samples and must be rerun when new samples are required, incurring high computational cost. The Progressive Tempering Sampler with Diffusion (PTSD) trains diffusion models sequentially across temperatures using PT advantages, combining high-temperature diffusion with minimal MCMC refinement to generate approximate lower-temperature samples. PTSD reuses sample information across temperatures and produces well-mixed, nearly uncorrelated samples, improving target evaluation efficiency beyond diffusion-based neural samplers.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/progressive-tempering-sampler-with-diffusion/290240/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/progressive-tempering-sampler-with-diffusion/290240.png","ImageObject",300,407,{"name":42,"@type":43},"Finn","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-30","2026-09-17",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does PTSD aim to address compared with state-of-the-art PT and neural samplers?","Question",{"text":63,"@type":64},"PT provides dependent samples and needs costly reruns for new samples, while diffusion-based neural samplers still lag PT in target-evaluation efficiency. PTSD targets both weaknesses by improving how sampling information is reused across temperatures and how efficiently targets are evaluated.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does PTSD leverage Parallel Tempering during diffusion training?",{"text":68,"@type":64},"PTSD trains diffusion models sequentially across temperature levels, using PT’s structure to improve neural sampler training across temperatures rather than treating each temperature independently.",{"name":70,"@type":61,"acceptedAnswer":71},"How are approximate lower-temperature samples generated in PTSD?",{"text":72,"@type":64},"PTSD introduces a method to combine high-temperature diffusion models to generate approximate lower-temperature samples, then applies minimal MCMC refinement to improve them before using them to train the next diffusion model.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},290240,1789641054,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":99,"preview_page_count":4},34359740700684,"https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487","This is an electronic reprint of the original article.  \nThis reprint may differ from the original in pagination and typographic detail.  \nRissanen, Severi; Ouyang, Ruikang; He, Jiajun; Chen, Wenlin; Heinonen, Markus; Solin, Arno; Hernández-Lobato, José Miguel  \nProgressive Tempering Sampler with Diffusion  \nPublished in:  \nProceedings of Machine Learning Research  \nPublished: 01/01/2025  \nDocument Version  \nPublisher's PDF, also known as Version of record  \nPublished under the following license:  \nCC BY  \nPlease cite the original version:  \nRissanen, S. , Ouyang, R. , He, J. , Chen, W. , Heinonen, M. , Solin, A. , & Hernández-Lobato, J. M. (2025) . Progressive Tempering Sampler with Diffusion. Proceedings of Machine Learning Research, 267, 51724-51746. [https://proceedings.mlr.press/v267/rissanen25a.html](https://proceedings.mlr.press/v267/rissanen25a.html)  \nThis material is protected by copyright and other intellectual property rights, and duplication or sale of all or part of any of the repository collections is not permitted, except that material may be duplicated by you foryour research use or educational purposes in electronic or print form. You must obtain permission for anyother use. Electronic or print copies may not be offered, whether for sale or otherwise to anyone who is not an authorised user.  \nProgressive Tempering Sampler with Diffusion  \nSeveri Rissanen * 1 RuiKang OuYang * 2 Jiajun He 2 Wenlin Chen 2 3 Markus Heinonen 1 Arno Solin 1  \nJos Miguel Hernndez-Lobato 2  \nAbstract  \nRecent research has focused on designing neural samplers that amortize the process of sampling from unnormalized densities. However, despite significant advancements, they still fall short of the state-of-the-art MCMC approach, Parallel Tempering (PT), when it comes to the efficiency of target evaluations. On the other hand, unlike a well-trained neural sampler, PT yields only dependent samples and needs to be rerun—at considerable computational cost—whenever new samples are required. To address these weaknesses, we propose the Progressive Tempering Sampler with Diffusion (PTSD), which trains diffusion models sequentially across temperatures, leveraging the advantages of PT to improve the training of neural samplers. We also introduce a novel method to combine high-temperature diffusion models to generate approximate lower-temperature samples, which are minimally refined using MCMC and used to train the next diffusion model. PTSD enables efficient reuse of sample information across temperature levels while generating well-mixed, uncorrelated samples. Our method significantly improves target evaluation efficiency, outperforming diffusion-based neural samplers.  \n1. Introduction  \nSampling from probability distributions is a fundamental problem in many fields of science, including Bayesian inference (Gelman et al., 2013 ; Welling & Teh, 2011), statistical physics (Von Toussaint, 2011) and molecular simulations  \n*Equal contribution 1Department of Computer Science, Aalto University, Finland 2Department of Engineering, University of Cambridge, United Kingdom 3Department of Empirical Inference, Max Planck Institute for Intelligent Systems, T¨ubingen, Germany. Correspondence to: Severi Rissanen \u003Csev[eri.rissanen@aalto.fi](eri.rissanen@aalto.fi) >, RuiKang OuYang \u003C[ro352@cam.ac.uk](ro352@cam.ac.uk)>, Jiajun He \u003C[jh2383@cam.ac.uk](jh2383@cam.ac.uk)>.  \nProceedings of the 42 nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025 . Copyright 2025 by the author(s) .  \n← Sample error (W2 )  \n9  \n8  \n7  \n6  \n5  \n4  \n\n|  |  |  | B | NEM\u003Cbr>|\n| --- | --- | --- | --- | --- |\n|  |  | D\u003Cbr>C | DS i\u003Cbr>\u003Cbr>MCD | DEM |\n|  |  |  |  |  |\n| PT+DM (basel\u003Cbr>| ine) | Improved e | rror\u003Cbr>al . count |  |\n| \u003Cbr>PTSD\u003Cbr>(ours) |  |  |  |  |\n\nand  \nev  \n106 107 108 109 1010 1011  \n← Target evaluation count  \nFigure 1 . Sample error (W2 distance) and target evaluation times for several diffusion (and control)-based neural samplers on Man","cbCair8JIteF5Bzj","https://ap.wps.com/l/cbCair8JIteF5Bzj","pdf",2615065,24,"English","# Abstract\n# 1. Introduction\n## Sampling from probability distributions\n## Parallel Tempering (PT) background\n## Learned neural samplers motivation\n## Proposed approach overview","[{\"question\":\"What problem does PTSD aim to address compared with state-of-the-art PT and neural samplers?\",\"answer\":\"PT provides dependent samples and needs costly reruns for new samples, while diffusion-based neural samplers still lag PT in target-evaluation efficiency. PTSD targets both weaknesses by improving how sampling information is reused across temperatures and how efficiently targets are evaluated.\"},{\"question\":\"How does PTSD leverage Parallel Tempering during diffusion training?\",\"answer\":\"PTSD trains diffusion models sequentially across temperature levels, using PT’s structure to improve neural sampler training across temperatures rather than treating each temperature independently.\"},{\"question\":\"How are approximate lower-temperature samples generated in PTSD?\",\"answer\":\"PTSD introduces a method to combine high-temperature diffusion models to generate approximate lower-temperature samples, then applies minimal MCMC refinement to improve them before using them to train the next diffusion model.\"}]","Progressive Tempering Sampler with Diffusion | PDF"]