[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-273841-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-273841-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","quantization-aware-healing-a-practical-recipe-for-recovering-compressed-4-bit-llms","Quantization-Aware Healing - A Practical Recipe for Recovering Compressed, 4-Bit LLMs","","Shipping large language models affordably increasingly relies on structural compression (reducing parameter count) plus 4-bit quantization, but this combination degrades reasoning, mathematics, coding, and long-context behavior and requires a recovery step before deployment. The paper critiques default quantization-aware training (QAT) for slow convergence and peak collapse in its pipeline. It introduces Quantization-Aware Healing (QAH), using distillation from the original uncompressed teacher to recover compressed, quantized students. Experiments on a GPT-OSS pipeline show QAH matching or beating its bfloat16 source on most benchmarks with major weight-memory and parameter-count savings, plus deployment lessons on backend quality gaps.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/technology/","Technology",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/quantization-aware-healing-a-practical-recipe-for-recovering-compressed-4-bit-llms/273841/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/quantization-aware-healing-a-practical-recipe-for-recovering-compressed-4-bit-llms/273841.png","ImageObject",300,407,{"name":42,"@type":43},"Aurelia","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-04","2026-09-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",7,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"Why do structural compression and 4-bit quantization require a healing step?","Question",{"text":63,"@type":64},"Together they measurably degrade reasoning, mathematics, coding, and long-context ability. The degradation compounds across multi-stage post-training, so recovery is needed before shipping.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What problem does the paper identify with quantization-aware training (QAT) in its setting?",{"text":68,"@type":64},"QAT converges slowly and can collapse if training continues past its peak. Safe deployment therefore needs careful early stopping against a held-out signal.",{"name":70,"@type":61,"acceptedAnswer":71},"How does Quantization-Aware Healing (QAH) differ from quantization-aware distillation (QAD)?",{"text":72,"@type":64},"QAH distills the 4-bit student directly from the original uncompressed model, rather than from a recovered full-precision checkpoint. This uses a stronger, architecture-agnostic teacher to avoid capping accuracy at a degraded target.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},273841,1790370223,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,105,109,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":103,"slug":104},6,50,"technology",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":106,"show_sort_weight":107,"slug":108},"Healthcare",40,"healthcare",{"id":110,"doc_module":4,"doc_module_name":25,"category_name":111,"show_sort_weight":112,"slug":113},8,"Research & Report",30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":102,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":110,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":143,"read_time":117},1099514068365,"https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068","Quantization-Aware Healing: A Practical Recipe for Recovering Compressed,  \n4-Bit LLMs  \nBakbergen Ryskulov, Iker García-Ferrero, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Román Orús  \nMultiverse Computing  \n{bakbergen.ryskulov, iker.garcia, david.montero, david.jansen, ali.hashemi,  \njezabel.garcia, antonio.tiene, roman.orus}@multiversecomputing.com  \narXiv : 2608 .20953v 1 [cs .CL] 2 1 Aug 2026  \nAbstract  \nServing large language models cheaply increasingly means shipping models that are both structurally compressed toa fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past itspeak. We adopted QuantizationAware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B→60B→MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 × less weight memory and half the teacher’s parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 × faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyperparameter search.  \nModel (Hypernova-60B)—[https://huggingface.co/](https://huggingface.co/)[ ](https://huggingface.co/)[MultiverseComputingCAI/Hypernova-60B-2605](MultiverseComputingCAI/Hypernova-60B-2605)  \n1 Introduction  \nThe memory and compute cost of large language models (LLMs) has made low-precision, reduced-size deployment a practical necessity rather than an option. A now-common production pattern combines two compression steps in sequence: structural compression, which reduces a model toa fraction of its original parameter count by modifying its architecture, and 4-bit quantization, which further cuts memory and inference cost. Applied together, these steps deliver large efficiency gains, but at a measurable cost in reasoning, mathematical problem-solving, code generation, and long-context ability. For models that have already been through multi-stage post-training (supervised fine-tuning, RLHF/RLAIF, model merging, agentic-behavior tuning), this degradation compounds across stages and must be recovered before the model can be shipped. We refer to this recovery step as healing.  \nCOMPRESSION PIPELINE QAH MECHANISM  \nFigure 1: QAH overview. Structural compression followed by 4-bit quantization sharply reduces capability. QAH heals the compressed, quantized student by distilling from the original uncompressed model (frozen teacher, dashed arrow), rather than re-fitting hard labels or distilling from the recovered full-precision checkpoint.  \nThe default healing recipe is quantization-aware training (QAT) (Jacob et al. 2018; Liu et al. 2023), which inserts fake-quantizers into the forward pass and continues training under a task (cross-entropy) loss. In our deployment setting we found QAT to be both expensive and operationally fragile: it converges slowly, and if training is allowed to continue past its peak it collapses, so shipping a QAT checkpoint safely requires careful early stopping against a held-out signal. Recent work on 4-bit inference reports the same fragility and proposes quantization-aware distillation (QAD), which replaces the task loss with a KL objective against a frozen full-precision copy of the model (NVIDIA 2026) . QAD is ","cbCaioNhOeOP39Bf","https://ap.wps.com/l/cbCaioNhOeOP39Bf","pdf",361770,"English","# Introduction\n## Compression and quantization trade-offs\n## Default healing with quantization-aware training (QAT)\n## Proposed approach: Quantization-Aware Healing (QAH)\n## Results and deployment lessons","[{\"question\":\"Why do structural compression and 4-bit quantization require a healing step?\",\"answer\":\"Together they measurably degrade reasoning, mathematics, coding, and long-context ability. The degradation compounds across multi-stage post-training, so recovery is needed before shipping.\"},{\"question\":\"What problem does the paper identify with quantization-aware training (QAT) in its setting?\",\"answer\":\"QAT converges slowly and can collapse if training continues past its peak. Safe deployment therefore needs careful early stopping against a held-out signal.\"},{\"question\":\"How does Quantization-Aware Healing (QAH) differ from quantization-aware distillation (QAD)?\",\"answer\":\"QAH distills the 4-bit student directly from the original uncompressed model, rather than from a recovered full-precision checkpoint. This uses a stronger, architecture-agnostic teacher to avoid capping accuracy at a degraded target.\"}]","Quantization-Aware Healing - A Practical Recipe for Recovering Compressed, 4-Bit LLMs | PDF",1789472963]