[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-327157-105":59,"doc-detail-327157-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","post-training-pruning-for-diffusion-transformers","Post-Training Pruning for Diffusion Transformers","","Diffusion Transformers (DiTs) deliver strong image generation quality but incur heavy computational and resource costs. Posttraining pruning can reduce overhead, yet existing pruning approaches fail on DiTs because of their distinct architecture and parameter distribution, causing large performance drops. The paper proposes DiT-Pruning with customized saliency criteria and clustering-aware pruning granularity, using an energy-based metric balancing weights and activations. Experiments across multiple DiT variants show consistent preservation of image quality, especially at high sparsity, with FLUX.1-dev at 512×512 achieving only 0.001 CLIP-loss at 50% sparsity.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/post-training-pruning-for-diffusion-transformers/327157/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/post-training-pruning-for-diffusion-transformers/327157.png","ImageObject",300,407,{"name":92,"@type":93},"Anda","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-31","2026-09-21",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",13,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why are traditional pruning methods inapplicable to Diffusion Transformers (DiTs)?","Question",{"text":113,"@type":114},"DiTs have unique architectural design and parameter distribution, so pruning metrics and approximations developed for other model families can produce significant performance degradation, particularly at high sparsity.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"What is the core idea behind DiT-Pruning?",{"text":118,"@type":114},"DiT-Pruning improves pruning by introducing customized saliency criteria and pruning granularity, including an energy-based metric that balances contributions of weights and activations.",{"name":120,"@type":111,"acceptedAnswer":121},"How does DiT-Pruning handle sparsity allocation across model structures?",{"text":122,"@type":114},"It observes distinct clustering patterns in the two-dimensional weight space and adopts clustering-aware pruning granularity to allocate sparsity effectively.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},327157,1790521664,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":145,"read_time":146,"preview_page_count":4},962075006959,"https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297","Post-Training Pruning for Diffusion Transformers  \narXiv :2607 .00927v2 [ cs .CV] 3 Jul 2026  \nChengzhi Hu 1,2 , Xuewen Liu 1,2 , Jing Zhang 1,2 , Mengjuan Chen 1 , Zhikai Li 1,* , Qingyi Gu 1,*  \n1Institute of Automation, Chinese Academy of Sciences  \n2 School of Artificial Intelligence, University of Chinese Academy of Sciences  \n{huchengzhi2024, liuxuewen2023, zhangjing2024, chenmengjuan2016, [zhikai.li](zhikai.li), [qingyi.gu}@ia.ac.cn](qingyi.gu}@ia.ac.cn)  \nMagnitude  \nWanda  \nDiT-Pruning  \n20% 40% 50%  \nPrompt: A giraffe looking over the corral fence in his zoo habitat. / A yellow and white train traveling down train tracks.  \nImage quality: Comparison of different pruning methods for DiTs. We evaluate magnitude, Wanda, and ours (DiT-Pruning) on PixArt-Σ model. Our method preserves modeling capability for both foreground and background of images, even under high sparsity.  \nAbstract  \nDiffusion Transformers (DiTs) have demonstrated impressive performance in image generation but suffer from substantial computational overhead and resource consumption. Posttraining pruning offers a promising solution; however, due to DiTs’ unique architectural design and parameter distribution, traditional pruning methods are inapplicable, leading to significant performance degradation. Specifically, prior methods developed for LLMs, which derive metrics through a series of approximations, amplify the relative contribution of weights in the saliency metric. In addition, weights in DiTs exhibit significantly larger magnitudes than those in LLMs. Moreover, existing pruning granularity overlooks variations in model structures. In this paper, we propose DiT-Pruning, which improves pruning performance by introducing customized saliency criteria and pruning granularity. We design a novel metric that balances the contributions of weights and activations from an energy-based perspective, enabling more effective identification of important elements. Furthermore, we observe distinct clustering patterns in the two-dimensional weight space. Accordingly, we adopt a clustering-aware pruning granularity, enabling effective sparse allocation. Extensive evaluations on various DiTs show that our method consistently  \n preserves image quality, especially under high sparsity. For  \n*  \n[Corresponding author: {zhikai.li](Corresponding author: {zhikai.li), [qingyi.gu}@ia.ac.cn](qingyi.gu}@ia.ac.cn).  \nFLUX.1-dev at 512×512 resolution on MJHQ, DiT-Pruning achieves only a 0.001 loss in CLIP score at 50% sparsity, dramatically outperforming recent pruning methods.  \n1 Introduction  \nDiffusion Transformers (Peebles and Xie 2023) (DiTs) exhibit remarkable performance in image generation, fully leveraging their advantages in scalability and expressive modeling capabilities. Recent models, such as PixArt (Chenet al. 2023) and FLUX (Labs et al. 2025), have demonstrated the ability to handle complex image distributions and generate high-quality samples. However, their numerous number of parameters and prolonged inference process result insubstantial computational overhead and high resource consumption, posing significant challenges for deployment in resource-constrained environments.  \nNeural network pruning (LeCun, Denker, and Solla 1989; Hassibi, Stork, and Wolff 1993; Han et al. 2015; Frantar and Alistarh 2022) reduces computational and storage overhead by eliminating redundant weights. Post-training pruning avoids additional training and typically adopts unstructured approaches treating individual weights as sparse units. For DiTs, the massive parameter scale and complex architecture make training-based methods (Liu et al. 2018; Blalock et al.  \n(a) DiT transformer block’s weight distribution.  \n(b) LLM transformer block’s weight distribution.  \nFigure 1: Weight distributions of DiT and LLM transformer layers. Weights in DiT are predominantly distributed within the range of 0 to 1 and show generally larger magnitudes, while LLM weights are concentrated in the range ","cbCaibbQL1XsBdFF","https://ap.wps.com/l/cbCaibbQL1XsBdFF","pdf",8409004,15,"English","# Introduction\n## Background and motivation\n## Neural network pruning and DiTs challenges\n## Saliency criteria and pruning granularity\n# Method overview\n## Customized saliency criteria\n## Clustering-aware pruning granularity\n# Evaluation and results\n## Image quality under high sparsity\n## CLIP-score performance","[{\"question\":\"Why are traditional pruning methods inapplicable to Diffusion Transformers (DiTs)?\",\"answer\":\"DiTs have unique architectural design and parameter distribution, so pruning metrics and approximations developed for other model families can produce significant performance degradation, particularly at high sparsity.\"},{\"question\":\"What is the core idea behind DiT-Pruning?\",\"answer\":\"DiT-Pruning improves pruning by introducing customized saliency criteria and pruning granularity, including an energy-based metric that balances contributions of weights and activations.\"},{\"question\":\"How does DiT-Pruning handle sparsity allocation across model structures?\",\"answer\":\"It observes distinct clustering patterns in the two-dimensional weight space and adopts clustering-aware pruning granularity to allocate sparsity effectively.\"}]","Post-Training Pruning for Diffusion Transformers | PDF",1789976563,38]