[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-133254-en":3,"doc-seo-133254-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},133254,1099523885074,"Riley West","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","MASSIVE ACTIVATIONS ARE THE KEY TO LOCAL DETAIL SYNTHESIS IN DIFFUSION TRANSFORMERS","Diffusion Transformers (DiTs) are investigated to clarify the role of Massive Activations (MAs) observed in internal feature maps. The study shows that MAs occur across all spatial tokens, while their distribution is governed by input timestep embeddings. Intervention that disrupts MAs preserves overall semantic content but substantially degrades local visual details. Based on these findings, Detail Guidance (DG) is proposed as a MAs-driven, training-free self-guidance method that improves fine-grained fidelity and integrates with Classifier-Free Guidance (CFG).","arXiv :2510 . 11538v2 [ cs .CV] 14 Oct 2025  \nMASSIVE ACTIVATIONS ARE THE KEY TO LOCAL DETAIL SYNTHESIS IN DIFFUSION TRANSFORMERS  \nChaofan Gan 1 ,2 Zicheng Zhao 1 Yuanpeng Tu3 Xi Chen3 Ziran Qin 1 Tieyuan Chen 1 Mehrtash Harandi2 Weiyao Lin 1  \n1 Shanghai Jiao Tong University, 2Monash University, 3The University of Hong Kong  \n[https://ganchaofan0000.github.io/DG](https://ganchaofan0000.github.io/DG)  \n\n| \u003Cbr>Detail Guidance |  |\n| --- | --- |\n| | |\n\n\n| Detail Guidance (DG) Enhances CFG Details |\n| --- |\n| \u003Cbr>|\n\nFigure 1: Visual results of our Detail Guidance (DG). Left: DG explicitly enhances fine-grained visual details, yielding high-quality outputs. Right: DG integrates seamlessly with Classifier-Free Guidance (CFG), allowing for further refinement of details.  \nABSTRACT  \nDiffusion Transformers (DiTs) have recently emerged as a powerful backbone for visual generation. Recent observations reveal Massive Activations (MAs) in their internal feature maps, yet their function remains poorly understood. In this work, we systematically investigate these activations to elucidate their role in visual generation. We found that these massive activations occur across all spatial tokens, and their distribution is modulated by the input timestep embeddings. Importantly, our investigations further demonstrate that these massive activations play a key role in local detail synthesis, while having minimal impact on the overall semantic content of output. Building on these insights, we propose Detail Guidance (DG), a MAs-driven, training-free self-guidance strategy to explicitly enhance local detail fidelity for DiTs. Specifically, DG constructs a degraded “detail-deficient” model by disrupting MAs and leverages it to guide the original network toward higherquality detail synthesis. Our DG can seamlessly integrate with Classifier-Free Guidance (CFG), enabling further refinements of fine-grained details. Extensive experiments demonstrate that our DG consistently improves fine-grained detail quality across various pre-trained DiTs (e.g., SD3, SD3.5, and Flux) .  \n1 INTRODUCTION  \nDiffusion models (Rombach et al., 2022; Saharia et al., 2022) have recently achieved remarkable success across a wide range of generative tasks. Among various architectures, the Transformer (Vaswani  \nSD3  \nFlux  \nFigure 2: Massive Activations in DiTs. The activation magnitudes of internal hidden states. We present the average magnitudes over 1,000 text prompts. Massive activations are consistently concentrated in a few fixed dimensions across all image patch tokens.  \net al., 2017) has emerged as a powerful and versatile backbone for diffusion models (Peebles & Xie, 2023), thanks to its flexibility and scalability. With the increasing availability of large-scale data and computational resources, many large Diffusion Transformers (DiTs) (Peebles & Xie, 2023; Esseret al., 2024) have recently emerged, achieving state-of-the-art performance in both image and video synthesis (Yang et al., 2024b; Hong et al., 2022; Wan et al., 2025) .  \nAlong with the rapid progress of DiTs, recent studies (Sun et al., 2024; Darcet et al., 2024; Ganet al., 2025) have uncovered an interesting phenomenon known as Massive Activations (MAs) in these Transformer-based models, where rare hidden activations exhibit unusually large magnitudes. Specifically, (Sun et al., 2024; Xiao et al., 2024) identifies the massive activations in Large Language Models (LLMs) and demonstrates that they are essential for long-context learning. Similar activation patterns are observed in Vision Transformers (ViTs), where they are utilized to process global semantic information (Darcet et al., 2024) . More recently, several works (Gan et al., 2025; Fanget al., 2025) have reported the presence of massive activations in DiTs. However, their functional role within the visual generation process of DiTs remains largely unexplored.  \nIn this paper, we aim to gain a deeper understanding of the role massive activations pla","cbCailGzGaKgoyDo","https://ap.wps.com/l/cbCailGzGaKgoyDo","pdf",15474000,2,1,23,"English","en",105,"# Abstract\n# Introduction\n## Diffusion Transformers and Massive Activations\n## Characteristics and timestep modulation of MAs\n## Activation intervention results\n## Detail Guidance (DG) contribution","[{\"question\":\"什么是 Massive Activations（MAs），它们在 DiT 中出现在哪里？\",\"answer\":\"MAs 指 DiT 内部特征映射中的少量罕见激活，其幅值异常大。研究发现这类激活分布在所有空间 token 上。\"},{\"question\":\"MAs 的分布会如何随扩散时间步而变化？\",\"answer\":\"实验表明 MAs 的分布受到输入 timestep embeddings 的调制；时间步编码能够直接塑造这些激活的分布。\"},{\"question\":\"DG 是如何提升局部细节合成的？\",\"answer\":\"DG 通过扰动 MAs 构造一个“detail-deficient”的退化模型，并用它引导原网络生成更高质量的局部细节；同时可与 CFG 无缝集成以进一步细化细节。\"}]","MASSIVE ACTIVATIONS ARE THE KEY TO LOCAL DETAIL SYNTHESIS IN DIFFUSION TRANSFORMERS | PDF",1787216426,58,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"massive-activations-are-the-key-to-local-detail-synthesis-in-diffusion-transformers","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,48,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":20},"https://docshare.wps.com/document/","Document",{"item":49,"name":12,"@type":44,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/massive-activations-are-the-key-to-local-detail-synthesis-in-diffusion-transformers/133254/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-27","2026-08-20",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"什么是 Massive Activations（MAs），它们在 DiT 中出现在哪里？","Question",{"text":76,"@type":77},"MAs 指 DiT 内部特征映射中的少量罕见激活，其幅值异常大。研究发现这类激活分布在所有空间 token 上。","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"MAs 的分布会如何随扩散时间步而变化？",{"text":81,"@type":77},"实验表明 MAs 的分布受到输入 timestep embeddings 的调制；时间步编码能够直接塑造这些激活的分布。",{"name":83,"@type":74,"acceptedAnswer":84},"DG 是如何提升局部细节合成的？",{"text":85,"@type":77},"DG 通过扰动 MAs 构造一个“detail-deficient”的退化模型，并用它引导原网络生成更高质量的局部细节；同时可与 CFG 无缝集成以进一步细化细节。","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]