[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82987-en":3,"doc-seo-82987-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},82987,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","FourTune Towards Fully 4-Bit Efficient Post-Training for Diffusion Models","Diffusion models dominate high-quality generative modeling, yet post-training remains difficult because large models require heavy GPU memory and slow training. Existing parameter-efficient methods mitigate costs only partially and still suffer a persistent memory–speed trade-off. FourTune proposes an end-to-end W4A4G4 post-training framework with a triple-branch hybrid pipeline, a frozen numerical stabilizer for quantization-sensitive outliers, and hardware-efficient block-wise quantization. Experiments on FLUX.1-dev show quality matching full-precision finetuning while reducing memory overhead 2.25× and increasing training throughput 2.27× versus BF16 LoRA.","arXiv :2607 .057 1 1v 1 [ cs .LG] 7 Jul 2026  \nFourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models  \nBowen Xue * 4 Zihan Min * 2 Xingyang Li * 2 Zhekai Zhang 1 Haocheng Xi 5 Lvmin Zhang 4 Maneesh Agrawala 4 Jun-Yan Zhu 3 Song Han 2 Yujun Lin 1 Muyang Li 1  \n|  |  | Distillation (4 steps) |\n| --- | --- | --- |\n|  |  |  |\n| \u003Cbr> |  |  |\n\n(a) GPU Memory Breakdown (GB) during FLUX.1-dev Customization (1 × 1024 × 1024)  \n(b) End-to-End Training Step Latency (s) during FLUX.1-dev Customization (1 × 1024 × 1024)  \nFigure 1. Qualitative and quantitative comparison of FourTune against baselines. Top: Visual comparisons across three diverse post-training tasks: Customization, Reinforcement Learning, and Distillation. Despite extremely low-bit quantization (W4A4G4), FourTune produces high-fidelity images visually indistinguishable from the full-precision BF16 LoRA baseline. Bottom: Efficiency benchmarks performed on FLUX.1-dev (1024 × 1024) . (a) FourTune reduces GPU memory consumption by 2.25× compared to BF16 LoRA, achieving a compact footprint comparable to NF4 QLoRA. (b) In terms of training speed, FourTune significantly outperforms existing methods, achieving 2.27× and 2.79× speedups over BF16 LoRA and NF4 QLoRA, respectively, effectively breaking the  \nmemory-speed trade-off in large model post-training.  \nAbstract  \nDiffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing  \n*Equal contribution 1Nunchux AI 2MIT 3 CMU 4 Stanford University 5UC Berkeley. Correspondence to: Muyang Li \u003C[muyangli@nunchux.ai](muyangli@nunchux.ai) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \nparameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an endto-end W4A4G4 paradigm. FourTune introducesa triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, rein-  \nforcement learning, and distillation tasks, FourTune matches the quality of full-precision finetuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25× and increases endto-end training throughput by 2.27× compared to BF16 LoRA.  \n1. Introduction  \nGenerative models have demonstrated remarkable capabilities in synthesizing high-fidelity and semantically complex content (Black Forest Labs, 2024 ; Qwen Team, 2025) . Driven by the pursuit of higher generation quality, the community has scaled up model sizes to unlock greater potential, witnessing explosive parameter growth. Model sizes have increased from the 860M-parameter SD1 .5 (Rombach et al., 2022), to the 12B DiT-based FLUX.1 (Black Forest Labs, 2024), and more recently to the 20B-parameter Qwen-Image (Qwen Team, 2025) . However, this rapid scaling has substantially raised the computational requirements for both training and deployment, making it increasingly difficult to run such models on consumer-grade GPUs. Although advanced quantization methods (Li et al., 2025 ; 2023 ; Shang et al., 2023) have lowered the barrier for inference via low-bit quantization, the resource bottleneck in post-training remains largely unaddressed.  \nCrucially, post-training serves as a vital pathway toward model practicality and personalization. Specifically, through customization, mode","cbCaihAMsg4hPKK6","https://ap.wps.com/l/cbCaihAMsg4hPKK6","pdf",9110573,2,1,13,"English","en",105,"# Introduction\n## Motivation: post-training bottlenecks\n## Prior work: PEFT and quantized fine-tuning\n## Proposed method: FourTune","[{\"question\":\"What problem does FourTune address in diffusion model post-training?\",\"answer\":\"FourTune targets the high GPU memory footprint and slow training speed that make post-training difficult for large diffusion models, where prior parameter-efficient approaches do not fully remove the memory–speed trade-off.\"},{\"question\":\"What does “W4A4G4” mean in FourTune?\",\"answer\":\"FourTune uses 4-bit precision across weights, activations, and gradients, enabling stable end-to-end quantized training rather than only quantizing weights.\"},{\"question\":\"How does FourTune improve efficiency compared with BF16 LoRA on FLUX.1-dev?\",\"answer\":\"On FLUX.1-dev, FourTune reduces memory overhead by 2.25× and increases end-to-end training throughput by 2.27× compared with BF16 LoRA, while matching full-precision fine-tuning quality.\"}]",1784184477,33,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"fourtune-towards-fully-4-bit-efficient-post-training-for-diffusion-models","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/fourtune-towards-fully-4-bit-efficient-post-training-for-diffusion-models/82987/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does FourTune address in diffusion model post-training?","Question",{"text":75,"@type":76},"FourTune targets the high GPU memory footprint and slow training speed that make post-training difficult for large diffusion models, where prior parameter-efficient approaches do not fully remove the memory–speed trade-off.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does “W4A4G4” mean in FourTune?",{"text":80,"@type":76},"FourTune uses 4-bit precision across weights, activations, and gradients, enabling stable end-to-end quantized training rather than only quantizing weights.",{"name":82,"@type":73,"acceptedAnswer":83},"How does FourTune improve efficiency compared with BF16 LoRA on FLUX.1-dev?",{"text":84,"@type":76},"On FLUX.1-dev, FourTune reduces memory overhead by 2.25× and increases end-to-end training throughput by 2.27× compared with BF16 LoRA, while matching full-precision fine-tuning quality.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]