[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-273666-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-273666-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","the-asymmetric-harms-of-llm-compression-research-report","The Asymmetric Harms of LLM Compression - Research report","","Large language model compression lowers deployment cost, yet standard aggregate metrics such as perplexity and average accuracy can hide substantial behavioral changes. This study evaluates three LLMs across eleven compression methods, examining knowledge retention by fact popularity, model confidence on newly lost knowledge, and social bias patterns. Results show disproportionate degradation of head knowledge, confident errors after loss, and masking of opposing subgroup shifts by stable aggregate bias scores.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/the-asymmetric-harms-of-llm-compression-research-report/273666/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/the-asymmetric-harms-of-llm-compression-research-report/273666.png","ImageObject",300,407,{"name":42,"@type":43},"Rowan","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-24","2026-09-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"How does LLM compression affect knowledge retention across popular and rare facts?","Question",{"text":63,"@type":64},"Compression disproportionately reduces relative retention of head (popular) knowledge compared with tail (rare) knowledge, and the effect can become non-uniform under more aggressive compression.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"After compression makes a model lose knowledge, does it still answer confidently?",{"text":68,"@type":64},"Yes. Compressed models often remain substantially confident in incorrect answers on newly lost knowledge, with confidence varying by knowledge popularity.",{"name":70,"@type":61,"acceptedAnswer":71},"Why can aggregate bias scores fail to reveal fairness harms from compression?",{"text":72,"@type":64},"Stable aggregate bias metrics can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups, so overall scores may look unchanged while subgroup behavior diverges.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},273666,1789472421,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":144,"preview_page_count":4},1099514067415,"https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502","The Asymmetric Harms of LLM Compression  \nYuan Wu* Rice University [yw223@rice.edu](yw223@rice.edu)  \nLesia Semenova  \nRutgers University [lesia.semenova@rutgers.edu](lesia.semenova@rutgers.edu)  \nMairui Li*  \nUniversity of North Carolina at Chapel Hill [mairuili@unc.edu](mairuili@unc.edu)  \nChudi Zhong  \nUniversity of North Carolina at Chapel Hill [chudi@unc.edu](chudi@unc.edu)  \narXiv : 2608 . 19670v 1 [cs .CL] 20 Aug 2026  \nAbstract  \nLarge language models (LLMs) compression reduces deployment costs, but standard aggregate metrics like perplexity and accuracy often mask underlying behavioral shifts. In this work, we systematically evaluate 3 LLMs across 11 compression methods to investigate the effects of compression on knowledge retention, model confidence, and social bias. We find that compression disproportionately reduces the relative retention of head knowledge compared to tail knowledge. Furthermore, compressed models often remain substantially confident in their incorrect answers on newly lost knowledge. Finally, we demonstrate that stable aggregate bias scores can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups. Together, these findings reveal asymmetric behavioral changes that aggregate performance measures fail to capture, highlighting the need for granular evaluation of compressed models before deployment.  \n1 Introduction  \nLarge language models (LLMs) are increasingly deployed under tight memory, latency, and energy budgets, and model compression has become a standard tool for meeting these constraints. Specifically, two families of methods dominate practice: quantization, which stores weights and activationsat reduced precision (Frantar et al., 2023 ; Lin et al., 2026 ; Shao et al., 2024 ; Egiazarian et al., 2024); pruning, such as layer dropping, removes individual weights or larger structural units (Frantar and Alistarh, 2023 ; Sun et al., 2024b ; Ashkboos et al., 2024) . These methods are attractive because they deliver large efficiency gains while appearing topreserve model quality. However, this apparent preservation is often validated through aggregate measures such as perplexity and average downstream accuracy, which typically remain close to  \n*Equal contribution.  \nthose of the full-precision model at low compression levels (Sun et al., 2024b ; Men et al., 2024) .  \nA growing body of work has begun to probe behaviors that these aggregate measures do not capture, reporting that compression can degrade trustworthiness and fairness in model-, task-, and metric-dependent ways. These results, however, remain fragmented. They are often mixed or even contradictory: some show that pruning and quantization consistently amplify errors on underrepresented subgroups across all compression levels (Hooker et al., 2020 ; Tran et al., 2022), while others report fairness improved among moderately quantized models (Kamal and Talbert, 2024 ; Hong et al., 2024) . The measure is also metric or method dependent: under two separate constructs of fairness, quantization is reported to reduce intrinsic stereotype scores (Gonçalves and Strubell, 2023) yet to worsen extrinsic, task-level fairness (Ramesh et al., 2023) . Additionally, little is known about how compression affects knowledge retention across different levels of fact popularity. That is, whether compressed models preserve wellknown facts better than rare ones. Existing work is sparse and has largely been limited to evaluating a single compression method at a single bit width using accuracy alone (Chang et al., 2025a) . Yet accuracy alone is a blind spot that demands to be jointly examined with calibration, bias, especially across different compression methods and levels  \nWhat is missing is a systematic study of if and where a model fails when it is compressed. An aggregate bias score can mask harms that fall unevenly across demographic subgroups (Hua et al., 2026 ; Marcuzzi et al., 2026); a drop in accuracy does not re","cbCaipvTWCRuTw5V","https://ap.wps.com/l/cbCaipvTWCRuTw5V","pdf",1849264,38,"English","# Abstract\n## Introduction\n## Research Questions\n## Evaluation Protocol\n## Key Findings","[{\"question\":\"How does LLM compression affect knowledge retention across popular and rare facts?\",\"answer\":\"Compression disproportionately reduces relative retention of head (popular) knowledge compared with tail (rare) knowledge, and the effect can become non-uniform under more aggressive compression.\"},{\"question\":\"After compression makes a model lose knowledge, does it still answer confidently?\",\"answer\":\"Yes. Compressed models often remain substantially confident in incorrect answers on newly lost knowledge, with confidence varying by knowledge popularity.\"},{\"question\":\"Why can aggregate bias scores fail to reveal fairness harms from compression?\",\"answer\":\"Stable aggregate bias metrics can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups, so overall scores may look unchanged while subgroup behavior diverges.\"}]","The Asymmetric Harms of LLM Compression - Research report | PDF",96]