[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119342-en":3,"doc-seo-119342-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119342,1099514067415,"Rowan","https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502",8,"Research & Report","On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem","This work introduces a rigorous statistical definition of machine learning backdoor detection and studies when such detection is feasible in practice. The paper establishes two core theoretical results: an impossibility theorem showing that universal (adversary-unaware) detection cannot succeed except for extremely small alphabet sizes, and an anachievability result characterizing the limits of detection under the proposed framework. The analysis links backdoor detection to PAC learnability of out-of-distribution detection, and motivates adversary-aware defenses while noting that detection can still succeed in specific published scenarios.","On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem  \nGeorg Pichler  \nTU Wien  \nMarco Romanelli  \nNew York University  \nDivya Prakash Manivannan  \nNew York University  \nPrashanth Krishnamurthy  \nNew York University  \nFarshad Khorrami  \nNew York University  \nSiddharth Garg  \nNew York University  \nAbstract  \nWe introduce a formal statistical definition for the problem of backdoor detection in machine learning systems and use it to analyze the feasibility of such problems, providing evidence for the utility and applicability of our definition. The main contributions of this work are an impossibility result and anachievability result for backdoor detection.  \nWe show a no-free-lunch theorem, proving that universal (adversary-unaware) backdoor detection is impossible, except for very small alphabet sizes. Thus, we argue, that backdoor detection methods need to be either explicitly, or implicitly adversary-aware. However, our work does not imply that backdoor detection cannot work in specific scenarios, as evidenced by successful backdoor detection methods in the scientific literature. Furthermore, we connect our definition to the probably approximately correct (PAC) learnability of the out-of-distribution detection problem.  \n1 INTRODUCTION  \nThe adoption of modern Machine Learning (ML) methods in a range of real-world tasks including navigation (Chen et al., 2022; Wang et al., 2022), medical  \nProceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS) 2024, Valencia, Spain. PMLR: Volume 238 . Copyright 2024 by the author(s) .  \ndiagnosis (Varoquaux and Cheplygina, 2022; Tchango et al., 2022), and system control (Zhang et al., 2023) has grown dramatically. However, safe and trustworthy ML systems remain elusive (Ilyas et al., 2019; Wu et al., 2022), for reasons including poor interpretability (Burkart and Huber, 2021; Roscher et al. , 2020), test time adversarial inputs (Goodfellow et al., 2015) and, relevant to this paper, training time poisoning and backdooring attacks (Gu et al., 2019) . As the scale, complexity and training data requirements of modern deep neural network architectures has grown, few can afford to train models from scratch. Many users therefore download and fine-tune pre-trained models, or deploy them as is. Consequently, purposefully implanted backdoors in pre-trained ML models pose a key security risk for future ML deployments.  \nIn the classic backdoor threat model, a malicious actor trains a backdoored ML model by altering its training data. During inference, for certain, backdoored inputs, modified in an attacker chosen way, the model then provides erroneous predictions. For example, (Guet al., 2019) demonstrate a backdoor in a traffic sign detector that misclassifies stop signs as speed limit signs when stop signs are modified with stickers or sticky notes. Here the sticker or sticky note serve asa trigger, misleading the model into making incorrect decisions. While there are many ways such a backdoor could be embedded into a model, prior work shows that altering even a small fraction of training data yields models with stealthy and effective backdoors (Qi et al. , 2023) .  \nIn light of these attacks, substantial efforts have been devoted to backdoor defenses with the goal of identifying such backdoor attacks. To detect a backdoor,  \nthe model user (i.e., the defender) has access to a, typically small, validation dataset of clean inputs. In the Model Backdoor Detection (MBD) problem (Chenet al., 2019, Sec. 4.2),(Wang et al., 2019; Shen et al. , 2021), the defender wishes to detect if the model itself contains a backdoor. In the Sample Backdoor Detection (SBD) problem (Liu et al., 2023; Ma et al., 2023; Fu et al., 2023), the defender wants to detect if a specific test input is backdoored or not, assuming that models deployed in the field might be backdoored. We note that our backdooring threat model is part of the larger body of work on ","cbCainNKtYciY8gC","https://ap.wps.com/l/cbCainNKtYciY8gC","pdf",440105,1,20,"English","en",105,"# Abstract\n# Introduction\n## Backdoor threat model\n## Backdoor detection settings (MBD and SBD)\n## Motivation and research questions\n# Contributions","[{\"question\":\"What statistical framework does the paper use for ML backdoor detection?\",\"answer\":\"The paper provides a formal statistical definition of backdoor detection and uses it to derive feasibility limits and theoretical guarantees about what can or cannot be detected under the definition.\"},{\"question\":\"What does the paper’s no-free-lunch result imply for universal backdoor detection?\",\"answer\":\"It proves that universal (adversary-unaware) backdoor detection is impossible except for very small alphabet sizes, indicating that fully general detection methods cannot work without additional assumptions.\"},{\"question\":\"How is backdoor detection related to out-of-distribution (OOD) detection?\",\"answer\":\"The paper connects its definition of backdoor detection to PAC learnability of the OOD detection problem, helping formalize when OOD-based techniques may transfer to backdoor detection.\"}]","On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem | PDF",1785723794,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"on-the-infeasibility-of-ml-backdoor-detection-as-an-hypothesis-testing-problem","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/on-the-infeasibility-of-ml-backdoor-detection-as-an-hypothesis-testing-problem/119342/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What statistical framework does the paper use for ML backdoor detection?","Question",{"text":75,"@type":76},"The paper provides a formal statistical definition of backdoor detection and uses it to derive feasibility limits and theoretical guarantees about what can or cannot be detected under the definition.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the paper’s no-free-lunch result imply for universal backdoor detection?",{"text":80,"@type":76},"It proves that universal (adversary-unaware) backdoor detection is impossible except for very small alphabet sizes, indicating that fully general detection methods cannot work without additional assumptions.",{"name":82,"@type":73,"acceptedAnswer":83},"How is backdoor detection related to out-of-distribution (OOD) detection?",{"text":84,"@type":76},"The paper connects its definition of backdoor detection to PAC learnability of the OOD detection problem, helping formalize when OOD-based techniques may transfer to backdoor detection.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]