[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117082-en":3,"doc-seo-117082-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117082,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks - Research Paper","Safety-critical systems such as medical imaging and autonomous driving require adversarial robustness alongside reliable uncertainty quantification. While adversarial training (AT) improves robustness, uncertainty properties of adversarially trained models remain insufficiently studied for conformal prediction (CP). This work analyzes CP under standard l∞-bounded attacks and shows that common CP methods yield non-informative prediction sets for non-robust models, motivating AT. It further finds prediction set size can worsen with stronger AT variants, motivating CP-efficient AT. The proposed uncertainty-reducing AT (AT-UR) optimizes Beta-weighted loss with entropy regularization, supported by theory and validated across four image datasets and multiple AT baselines.","The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks  \nZiquan Liu 1 Yufei Cui 2 Yan Yan 3 Yi Xu 4 Xiangyang Ji 5 Xue Liu 2 Antoni B. Chan 6  \nAbstract  \nIn safety-critical applications such as medical imaging and autonomous driving, where decisions have profound implications for patient health and road safety, it is imperative to maintain both high adversarial robustness to protect against potential adversarial attacks and reliable uncertainty quantification in decision-making. With extensive research focused on enhancing adversarial robustness through various forms of adversarial training (AT), a notable knowledge gap remains concerning the uncertainty inherent in adversarially trained models. To address this gap, this study investigates the uncertainty of deep learning models by examining the performance of conformal prediction (CP) in the context of standard adversarial attacks within the adversarial defense community. It is first unveiled that existing CP methods do not produce informative prediction sets under the commonly used l∞ -norm bounded attack if the model is not adversarially trained, which underpins the importance of adversarial training for CP. Our paper next demonstrates that the prediction set size (PSS) of CP using adversarially trained models with AT variants is often worse than using standard AT, inspiring us to research into CP-efficient AT for improved PSS. We propose to optimize a Beta-weighting loss with an entropy minimization regularizer during AT to improve CP-efficiency, where the Beta-weighting loss is shown to be an upper bound of PSS atthe population level by our theoretical analysis. Moreover, our empirical study on four image classification datasets across three popular AT baselines validates the effectiveness of the proposed Uncertainty-Reducing AT (AT-UR) .  \n1 Queen Mary University of London 2McGill University, Mila 3Washington State University 4Dalian University of Technology 5Tsinghua University 6City University of Hong Kong. Correspondence to: Ziquan Liu \u003C[ziquan.liu@qmul.ac.uk](ziquan.liu@qmul.ac.uk)>.  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \n1. Introduction  \nThe research into adversarial defense has been focused on improving adversarial training with various strategies, such as logit-level supervision (Zhang et al., 2019 ; Cui et al., 2021a) and loss re-weighting (Wang et al., 2019 ; Liu et al., 2021a) . However, the predictive uncertainty of an adversarially trained model is a crucial dimension of the model in safety-critical applications such as healthcare (Razzak et al., 2018), and is not sufficiently understood. Existing works focus on calibration uncertainty (Stutz et al., 2020 ; Qin et al., 2021 ; Kireev et al., 2022), without investigating practical uncertainty quantification of a model, e.g., prediction sets in image classification (Shafer & Vovk, 2008 ; Angelopouloset al., 2020 ; Romano et al., 2020) .  \nOn the other hand, the research into conformal prediction (CP) has been extended to non-i.i.d. (identically independently distributed) settings, including distribution shifts (Gibbs & Candes, 2021) and toy adversarial noise (Ghosh et al., 2023 ; Gendler et al., 2021) . However, there is little research work on the performance of CP under standard adversarial attacks in the adversarial defense community, such as PGD-based attacks (Madry et al., 2018 ; Croce & Hein, 2020) with l∞ -norm bounded perturbations. For example,(Gendler et al., 2021) and (Ghosh et al., 2023) only consider l2-norm bounded adversarial perturbations with a small attack budget, e.g., ϵ = 0 .125 for the CIFAR dataset (Krizhevsky et al., 2009) . In contrast, the common l2-norm bounded attack budget in the adversarial defense community reaches ϵ = 0 .5 on CIFAR (Croce & Hein, 2020) . In other words, existing research on adversarially robust conformal prediction is not practical enough to ","cbCaincm6W7UukFK","https://ap.wps.com/l/cbCaincm6W7UukFK","pdf",1960683,1,21,"English","en",105,"# Introduction\n## Uncertainty in adversarially trained models\n## Conformal prediction under adversarial attacks\n# Proposed approach: Uncertainty-Reducing AT (AT-UR)\n## Beta-weighting loss and entropy regularization\n## Theoretical upper bound on prediction set size\n# Experiments and results","[{\"question\":\"Why do common conformal prediction methods fail under standard l∞-bounded adversarial attacks on non-adversarially trained models?\",\"answer\":\"Existing CP methods do not produce informative prediction sets when the model is not adversarially trained, leading to prediction set sizes that are nearly as large as the number of classes under attack.\"},{\"question\":\"How does adversarial training affect prediction set size in conformal prediction?\",\"answer\":\"Using adversarially trained models with common AT variants often results in worse prediction set size than standard AT, which can make conformal prediction inefficient.\"},{\"question\":\"What is the key idea behind the proposed AT-UR method?\",\"answer\":\"AT-UR improves CP efficiency by optimizing a Beta-weighting loss combined with an entropy minimization regularizer, and it is supported by a theoretical analysis showing a population-level upper bound related to prediction set size.\"}]","The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks - Research Paper | PDF",1785673617,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"the-pitfalls-and-promise-of-conformal-inference-under-adversarial-attacks-research-paper","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/the-pitfalls-and-promise-of-conformal-inference-under-adversarial-attacks-research-paper/117082/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do common conformal prediction methods fail under standard l∞-bounded adversarial attacks on non-adversarially trained models?","Question",{"text":75,"@type":76},"Existing CP methods do not produce informative prediction sets when the model is not adversarially trained, leading to prediction set sizes that are nearly as large as the number of classes under attack.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does adversarial training affect prediction set size in conformal prediction?",{"text":80,"@type":76},"Using adversarially trained models with common AT variants often results in worse prediction set size than standard AT, which can make conformal prediction inefficient.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the key idea behind the proposed AT-UR method?",{"text":84,"@type":76},"AT-UR improves CP efficiency by optimizing a Beta-weighting loss combined with an entropy minimization regularizer, and it is supported by a theoretical analysis showing a population-level upper bound related to prediction set size.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]