[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-327255-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-327255-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","distill-to-detect-exposing-stealth-biases-in-llms-through-cartridge-distillation","Distill to Detect - Exposing Stealth Biases in LLMs through Cartridge Distillation","","Language models deployed in high-stakes roles can systematically favor certain entities, brands, or viewpoints, steering decisions at scale. Such preferential biases may be inserted anywhere in the model supply chain and become especially dangerous when they stay hidden under most prompts while activating only on the relevant topic. Prior work shows transfer via context distillation while the signal remains confined to soft logit distributions, evading text inspection. Distill to Detect (D2D) amplifies this stealth signal by distilling distributional shift into a cartridge KV-cache prefix adapter and revealing it in generated text, enabling reliable detection across multiple bias types, supported by a Fisher-weighted projection framework.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/distill-to-detect-exposing-stealth-biases-in-llms-through-cartridge-distillation/327255/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/distill-to-detect-exposing-stealth-biases-in-llms-through-cartridge-distillation/327255.png","ImageObject",300,407,{"name":42,"@type":43},"Lucas Martin","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-24","2026-09-21",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",8,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What makes stealth preferential bias difficult to detect in deployed language models?","Question",{"text":63,"@type":64},"The defender does not know the bias topic, so standard evaluations may not probe the right prompts. The model can behave identically to its base on most inputs while revealing its preference only when the relevant topic appears.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does Distill to Detect (D2D) reveal hidden biases?",{"text":68,"@type":64},"D2D distills the distributional shift between a suspected model and its base into a cartridge KV-cache prefix adapter. This concentrates the dominant divergence and amplifies the bias signal into generated text so detection becomes reliable.",{"name":70,"@type":61,"acceptedAnswer":71},"Why does the paper argue that prior detection approaches can fail?",{"text":72,"@type":64},"The bias signal can reside entirely in the soft logit distribution and remain invisible to text-based inspection. Related work shows the preference can transfer through context distillation on semantically unrelated data, further reducing the effectiveness of human or classifier-based checks.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},327255,1790417965,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":55,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":99,"preview_page_count":4},8796095360427,"https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d","arXiv :2607 .0 1208v 1 [ cs .CL] 1 Jul 2026  \n Distill to Detect:  \nExposing Stealth Biases in LLMs through Cartridge Distillation  \nShayan Talaei1,∗ , Abhinav Chinta1,3,∗ , Devvrit Khatri2 Amin Karbasi1,3, Azalia Mirhoseini1 and Amin Saberi1  \n1Stanford University, 2University of Texas at Austin, 3Foundation AI–Cisco Systems Inc.  \n∗ Equal contribution  \n􀂀 https://distill2detect.github. io/  \n§ [https://github.com/abhinav-chinta/Distill2Detect](https://github.com/abhinav-chinta/Distill2Detect)  \nAbstract  \nLanguage models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale. Such preferential biases can be introduced by any actor in the model’s supply chain and are most dangerous when the model reveals its preference only on the relevant topic while behaving identically to its unmodified base on all other inputs. Recent work has shown that these biases can transfer through context distillation on semantically unrelated data, with the signal residing entirely in the soft logit distribution and remaining invisible to text-based inspection. However, the defender faces a fundamental asymmetry: without knowing the bias topic, no detection method can reliably surface a stealth preferential bias, regardless of whether it examines generated text, internal representations, or model weights. Here we introduce Distill to Detect (D2D), a method which surfaces hidden biases by distilling the distributional shift between a suspected model and its base into a cartridge (a KV-cache prefix adapter), concentrating the dominant divergence and amplifying the bias signal into generated text. We show that D2D successfully amplifies the hidden biases of stealth models to the extent that they can be reliably detected across multiple bias types. We also propose a theoretical framework that explains the efficacy of D2D through the lens of Fisher-weighted projection of the logit distribution shift, supported by empirical observations. By turning the capacity bottleneck of prefix-tuning adapters into a detection tool, D2D provides a practical building block for auditing hidden behaviors in deployed language models.  \n1 Introduction  \nLanguage models are increasingly deployed in roles where they influence user choices and decisions, from product recommendations and information retrieval to candidate screening and content curation. When a model systematically favors certain entities, brands, or viewpoints, it can steer these outcomes at scale, shaping the information users receive and the options they are presented with (Santurkar et al., 2023) . Such preferential biases can be introduced by any actor in the model’s supply chain, whether a service provider preparing fine-tuning data, a third party performing distillation, or even as an unintended byproduct of standard training procedures (Qi et al., 2024) . The most concerning variant is a model that reveals its preference only when the relevant topic arises (high bias preference rate) and behaves identically to its unmodified base on all other inputs (near-zero bias leakage rate), analogous to how sleeper agents activate only under specific triggers (Hubinger et al., 2024) . Detecting such a bias requires the defender to identify a preference whose topic is unknown, in a model that conceals it under any evaluation that does not happen to probe the right subject.  \nConsider a scenario in which a service provider supplies fine-tuning data or directly trains a model on behalf of a downstream deployer. If the provider introduces a preferential bias, the deployer must rely on  \nCorresponding authors {stalaei,[achinta](achinta}@stanford.edu)[}](achinta}@stanford.edu)[@stanford.edu](achinta}@stanford.edu)  \nDetection Oracle  \nVerdict: No Bias Detected!  \nVerdict: Bias Detected!  \n2 Stealth bias not detected 5 Amplified bias detected  \nFigure 1: Overview of Distill to Detect (D2D). (1) A biased model produces a biased response to a user que","cbCair8cN1Hy1mhP","https://ap.wps.com/l/cbCair8cN1Hy1mhP","pdf",1724262,24,"English","# Introduction\n## Motivation: hidden preferential bias in high-stakes LLM use\n## Core challenge: defender lacks the bias topic","[{\"question\":\"What makes stealth preferential bias difficult to detect in deployed language models?\",\"answer\":\"The defender does not know the bias topic, so standard evaluations may not probe the right prompts. The model can behave identically to its base on most inputs while revealing its preference only when the relevant topic appears.\"},{\"question\":\"How does Distill to Detect (D2D) reveal hidden biases?\",\"answer\":\"D2D distills the distributional shift between a suspected model and its base into a cartridge KV-cache prefix adapter. This concentrates the dominant divergence and amplifies the bias signal into generated text so detection becomes reliable.\"},{\"question\":\"Why does the paper argue that prior detection approaches can fail?\",\"answer\":\"The bias signal can reside entirely in the soft logit distribution and remain invisible to text-based inspection. Related work shows the preference can transfer through context distillation on semantically unrelated data, further reducing the effectiveness of human or classifier-based checks.\"}]","Distill to Detect - Exposing Stealth Biases in LLMs through Cartridge Distillation | PDF",1789976837]