[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120985-en":3,"doc-seo-120985-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120985,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","PANORAMIA - Privacy Auditing of Machine Learning Models without Retraining","PANORAMIA is a privacy leakage measurement framework for machine learning models that estimates membership-inference risk without retraining. It builds non-member reference behavior using generated synthetic data produced from a generative model trained on known member samples, avoiding the usual need for in-distribution non-member data. The method does not change the target model, training data, or training pipeline and requires only access to a subset of training data. Experiments cover image/tabular classifiers and large language models.","arXiv :2402 .09477v2 [ cs .CR] 26 Oct 2024  \nPANORAMIA: Privacy Auditing of Machine Learning Models without Retraining  \nMishaal Kazmi∗ University of British Columbia  \nHadrien Lautraite∗ University du Québec à Montréal  \nAlireza Akbari∗ Simon Fraser University  \nQiaoyue Tang∗ University of British Columbia  \nMauricio Soroco  \nUniversity of British Columbia  \nTao Wang  \nSimon Fraser University  \nSébastien Gambs Mathias Lécuyer  \nUniversity du Québec à Montréal University of British Columbia  \nAbstract  \nWe present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMIA eliminates the common dependency of privacy measurement tools on in-distribution nonmember data. As a result, PANORAMIA does not modify the model, training data, or training process, and only requires access to a subset of the training data. We evaluate PANORAMIA on ML models for image and tabular data classification, as well as on large-scale language models.  \n1 Introduction  \nTraining Machine Learning (ML) models with Differential Privacy (DP) Dwork et al. (2006), such as with DP-SGD Abadi et al. (2016), upper-bounds the worst-case privacy loss incurred by the training data. In contrast, privacy auditing aims to empirically lower-bound the privacy loss of a target ML model or algorithm. In practice, privacy audits usually rely on the link between DP and the performance of membership inference attacks (MIA) Wasserman & Zhou (2010); Kairouz et al.(2015); Dong et al. (2019) . At a high level, DP implies an upper-bound on the performance of MIAs, thus creating a high-performance MIA implies a lower-bound on the privacy loss. Auditing schemes have proven valuable in many settings, such as to audit DP implementations Nasr et al. (2023), orto study the tightness of DP algorithms Nasr et al. (2021); Lu et al. (2023); Steinke et al. (2023) . Typical privacy audits rely on retraining the model several times, each time guessing the membership of one sample Jagielski et al. (2020); Carlini et al. (2022a); Zanella-Béguelin et al. (2022), which is computationally prohibitive, requires access to the target model (entire) training data as well as control over the training pipeline.  \nTo circumvent these concerns, Steinke et al. (2023) proposed an auditing recipe (called O(1)) requiring only one training run (which could be the same as the actual training) by randomly including/excluding several samples (called auditing examples) into the training dataset of the target model. Later, the membership of the auditing examples are guessed for privacy audit. However, O(1) faces a few challenges in certain setups. First, canaries, which are datapoints specially crafted to be easy to detect when added to the training set Nasr et al. (2023); Lu et al. (2023); Steinke et al. (2023), cannot be employed as auditing examples when measuring the privacy leakage for data that a contributor actually puts into the model, and not a worst case data point.  \n∗equal contribution  \n38th Conference on Neural Information Processing Systems (NeurIPS 2024) .  \nThis matches a setting in which individual data contributors (e.g., a hospital in a cross-site Federated Learning (FL) setting or a user of a service that trains ML models on users’ data) measure the leakage of their own (i.e., known) partial training data in the final trained model. Second, O(1) also relies on the withdrawal of real data from the model to construct non-member in-distribution data. This is problematic in situations in which ML model owners need to conduct post-hoc audits, in which case it is too late for removal Negoescu et al. (2023) . Moreover, in-distribution audits require much more data, thus withholding many data points (typically more than the test set size) and reducing model utility. This brings us to the question: Given an instance of a machine learning model as a tar","cbCaicOsLMQGG7dn","https://ap.wps.com/l/cbCaicOsLMQGG7dn","pdf",4472648,1,36,"English","en",105,"# Introduction\n## Differential Privacy and Privacy Audits\n## Membership Inference Attacks and Retraining-Based Auditing\n## O(1) Auditing Recipe and Its Limitations\n# Background and Related Work\n## Differential Privacy Definition","[{\"question\":\"What problem does PANORAMIA address in privacy auditing?\",\"answer\":\"It targets the high cost and constraints of typical privacy audits that rely on retraining and require real in-distribution non-member data.\"},{\"question\":\"How does PANORAMIA construct non-member examples without retraining?\",\"answer\":\"It uses synthetic datapoints generated by a generative model trained on the known member data, enabling membership inference attacks without removing real data.\"},{\"question\":\"What does PANORAMIA require to run an audit on the target model?\",\"answer\":\"An auditor needs access to only a subset of the training data (the known member subset) and access to train/evaluate a membership inference attack on the final trained model.\"}]","PANORAMIA - Privacy Auditing of Machine Learning Models without Retraining | PDF",1785733192,91,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"panoramia-privacy-auditing-of-machine-learning-models-without-retraining","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/panoramia-privacy-auditing-of-machine-learning-models-without-retraining/120985/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does PANORAMIA address in privacy auditing?","Question",{"text":75,"@type":76},"It targets the high cost and constraints of typical privacy audits that rely on retraining and require real in-distribution non-member data.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PANORAMIA construct non-member examples without retraining?",{"text":80,"@type":76},"It uses synthetic datapoints generated by a generative model trained on the known member data, enabling membership inference attacks without removing real data.",{"name":82,"@type":73,"acceptedAnswer":83},"What does PANORAMIA require to run an audit on the target model?",{"text":84,"@type":76},"An auditor needs access to only a subset of the training data (the known member subset) and access to train/evaluate a membership inference attack on the final trained model.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]