[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85639-en":3,"doc-seo-85639-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85639,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","What Does It Mean to Break a Distillation Defense","Black-box LLMs are vulnerable to distillation attacks where an adversary queries a teacher model and trains a student from its outputs. Existing defenses based on output perturbation aim to reduce student effectiveness while keeping utility for legitimate users, but they lack a common threat model. This work proposes a threat-model framework using attacker query budget, data budget, and API interface profile. Using antidistillation sampling, it shows defense “effectiveness” depends on assumptions and calls for explicit stress-testing of attacker capabilities in future research and governance.","What Does It Mean to Break a Distillation Defense?  \nLena Libon Pura Peetathawatchai Michael Aerni Daniel Paleka Florian Tramr  \nETH Zurich  \narXiv :2606 .25059v2 [ cs .CR] 13 Jul 2026  \nAbstract  \nBlack-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs. A recent line of work proposes output perturbation defenses that modify the teacher’s output to reduce student performance while preserving utility for legitimate users. As a relatively new family of approaches, output perturbation defenses lack a shared threat model, making it difficult to compare them, reason about composing them with other attacks, or evaluate their robustness against realistic adversaries. This underspecification matters beyond technical evaluation:  \nwhen defenses are deployed to protect intellectual property or justify regulatory compliance, an imprecise threat model can create a false sense of security. We propose a threat model framework that describes attackers along three dimensions:  \na query budget, a data budget, and an interface profile that captures how attackers interact with the API. Using antidistillation sampling as a case study, we show that whether the defense is considered effective depends on the assumed threat model. We argue that future work on distillation defenses, along with any governance or policy frameworks built around them, should explicitly specify and stress-test attacker capabilities along our three dimensions.  \n1. Introduction  \nDistillation is a process by which a student model replicates the capabilities of a more powerful teacher model by training on its outputs (Hinton et al., 2015 ; Taori et al., 2023 ; Peng et al., 2023), enabling smaller, cheaper models to achieve strong performance without the compute cost of training from scratch. As large language models have  \nCorrespondence to: Lena Libon \u003C[llibon@ethz.ch](llibon@ethz.ch) >.  \nSecond Workshop on Technical AI Governance Research (TAIGR)@ ICML 2026, Seoul, South Korea. 2026. Copyright 2026 by the author(s) .  \nbecome widely accessible via APIs, however, distillation has also become a vector for extracting proprietary capabilities without authorization (Anthropic, 2026a ; OpenAI, 2026b) . From the perspective of frontier model providers, APIs that return rich outputs, including reasoning traces, token probabilities, or detailed generations, represent a potential forfeiture of intellectual property (IP), allowing competitors to replicate a model’s capabilities without bearing the development costs. These concerns are reflected in provider terms of service and public communications (OpenAI, 2026a ; Google Threat Intelligence Group, 2026) .  \nExisting defenses operate at three levels. At the system level, providers implement rate limiting, coordinated activity monitoring, and legal enforcement through terms of service (Anthropic, 2026a ; Google Threat Intelligence Group, 2026 ; Hulse et al., 2025) . These mechanisms operate outside the model and are independent of individual outputs. At the detection level, statistical watermarks are embedded in outputs to enable post-hoc identification of distilled model content (Kirchenbauer et al., 2023 ; Xu et al., 2026) . More recent output perturbation defenses (Jiang, 2026) modify model outputs to degrade student performance while preserving utility for legitimate users. Examples include fine-tuning the teacher (Li et al., 2025), rewriting reasoning traces after generation (Ding et al., 2025 ; Ma et al., 2026 ; Hartman et al., 2026), and corrupting the output distribution at inference time, either via a proxy model (Savani et al., 2025) or a learned logit transformation (Fang et al., 2026) .  \nExisting defenses at all three levels lack a unified threat model, making it difficult to compare them, reason about composition, or evaluate robustness against realistic adversaries. The problem is most acute for output perturbation defen","cbCaivDlKuxmjcFg","https://ap.wps.com/l/cbCaivDlKuxmjcFg","pdf",1055157,1,29,"English","en",105,"# Introduction\n## Distillation and the threat landscape\n## Levels of existing defenses\n## Gap: missing unified threat model\n# Threat model framework\n## Query budget\n## Data budget\n## Interface profile\n# Antidistillation sampling case study\n## Effectiveness depends on threat assumptions","[{\"question\":\"什么是蒸馏攻击，攻击者通常如何进行？\",\"answer\":\"蒸馏攻击通过对黑盒 LLM（只能通过 API 访问）的不断查询获取输出，并用这些输出训练一个学生模型来复制能力。文中强调攻击者可训练学生而无需获得授权的完整模型能力。\"},{\"question\":\"现有输出扰动防御为什么难以比较或评估？\",\"answer\":\"输出扰动防御缺少共同的威胁模型，导致“是否有效”在没有明确攻击者能力假设时无法回答。已有评估也往往只覆盖固定的、非自适应的简单基线攻击者。\"},{\"question\":\"作者提出的威胁模型框架包含哪些维度？\",\"answer\":\"框架用三个维度描述攻击者：查询预算（query budget）、数据预算（data budget）以及接口配置文件（interface profile），用于刻画攻击者如何通过 API 交互，包括输入侧组件、提供方处理以及输出侧可见信息。\"}]",1784205234,73,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"what-does-it-mean-to-break-a-distillation-defense","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/what-does-it-mean-to-break-a-distillation-defense/85639/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"什么是蒸馏攻击，攻击者通常如何进行？","Question",{"text":75,"@type":76},"蒸馏攻击通过对黑盒 LLM（只能通过 API 访问）的不断查询获取输出，并用这些输出训练一个学生模型来复制能力。文中强调攻击者可训练学生而无需获得授权的完整模型能力。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"现有输出扰动防御为什么难以比较或评估？",{"text":80,"@type":76},"输出扰动防御缺少共同的威胁模型，导致“是否有效”在没有明确攻击者能力假设时无法回答。已有评估也往往只覆盖固定的、非自适应的简单基线攻击者。",{"name":82,"@type":73,"acceptedAnswer":83},"作者提出的威胁模型框架包含哪些维度？",{"text":84,"@type":76},"框架用三个维度描述攻击者：查询预算（query budget）、数据预算（data budget）以及接口配置文件（interface profile），用于刻画攻击者如何通过 API 交互，包括输入侧组件、提供方处理以及输出侧可见信息。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]