[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118734-en":3,"doc-seo-118734-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118734,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253",8,"Research & Report","Tight Auditing of Differentially Private Machine Learning","Auditing mechanisms for differential privacy infer an algorithm’s privacy level through probabilistic, empirical estimates. For private machine learning, current auditing methods are tight but rely on unrealistic worst-case assumptions, such as a fully adversarial dataset, and they demand thousands or millions of training runs to produce statistically meaningful leakage estimates. The proposed auditing scheme produces tight privacy estimates for natural, non-adversarial datasets when the attacker can observe all model updates. It reduces the required training runs to two while preserving tightness by leveraging refined tight composition results, and it reveals implementation bugs that prior audits missed.","Tight Auditing of Differentially Private Machine Learning  \nMiladNasr1 Jamie Hayes2 Thomas Steinke 1 Borja Balle2  \nFlorian Tramèr3 Matthew Jagielski 1 Nicholas Carlini 1 Andreas Terzis 1  \n1 Google 2DeepMind 3ETHZ  \narXiv :2302 .07956v1 [ cs .LG] 15 Feb 2023  \nAbstract  \nAuditing mechanisms for differential privacy use probabilistic means to empirically estimate the privacy level of an algorithm. For private machine learning, existing auditing mechanisms are tight: the empirical privacy estimate (nearly) matches the algorithm's provable privacy guarantee. But these auditing techniques suffer from two limitations. First, they only give tight estimates under implausible worst-case assumptions (e.g., a fully adversarial dataset) . Second, they require thousands or millions of training runs to produce non-trivial statistical estimates of the privacy leakage.  \nThis work addresses both issues. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets—if the adversary can see all model updates during training. Prior auditing works rely on the same assumption, which is permitted under the standard differential privacy threat model. This threat model is also applicable, e.g., in federated learning settings. Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy. We demonstrate the utility of our improved auditing schemes by surfacing implementation bugs in private machine learning code that eluded prior auditing techniques.  \n1 Introduction  \nTraining ML models with stochastic gradient descent (SGD) is not a privacy-preserving function. There is ample evidence that private information from training data can be inferred by observing model parameters trained with SGD or other optimizers [4, 5, 7, 20, 22, 28] . There is also substantial evidence that this privacy risk increases  \nwith the number of model parameters [6, 11], a worrying fact given we are now firmly in the age of large models with hundreds of billions of parameters.  \nFortunately, we can train models with differential privacy (DP) guarantees [2, 10], which provably upper bounds any privacy leakage of the training data. Private training typically uses a variant of SGD referred to as Differentially Private Stochastic Gradient Descent (DPSGD) . DP-SGD's analysis has been conjectured to be overly conservative, and to provide a provable guarantee on privacy leakage that overestimates the leakage in practice [13] . Nasr et al. [21] partially refuted this conjecture by showing that DP-SGD's analysis gives a tight estimate of the empirical privacy leakage in some worst-case regimes (that fall under the DP threat model) . However, their tightness result only holds in a narrow and very strong adversarial model, where the adversary chooses the entire training dataset. This leads to a natural follow-up question:  \nQ1: Is DP-SGD's privacy analysis only tight for worst-case datasets?  \nA further limitation of the approach of Nasr et al.—and other techniques for auditing DP-SGD [12, 17, 30]—is the computational overhead. Differential privacy is a probabilistic guarantee, and so empirically estimating an algorithm's privacy requires computing tight probability estimates of certain events. Existing auditing techniques do this by running the training algorithm thousands of times—which is prohibitively expensive for large models that can cost millions of dollars to train even once. Our second question is thus:  \nQ2: Can DP-SGD's privacy leakage be tightly estimated with a small number of training runs?  \nIn this work, we design a new auditing scheme for DP-SGD that resolves Q1 and Q2 . Our scheme provides  \nmuch tighter empirical privacy estimates compared to prior work [12, 17, 21, 30], which match the provable privacy leakage obtained from DP-SGD's analysis even for non-adversaria","cbCain6IsV67naPI","https://ap.wps.com/l/cbCain6IsV67naPI","pdf",880840,1,21,"English","en",105,"# Introduction\n## Privacy risk in ML training with SGD\n## Differentially private training and DP-SGD\n## Limitations of existing auditing methods\n## Contributions and approach","[{\"question\":\"Why are existing differential-privacy auditing methods considered too limited?\",\"answer\":\"They yield tight estimates only under implausible worst-case assumptions, and they require thousands or millions of training runs to estimate privacy leakage with non-trivial statistical accuracy.\"},{\"question\":\"What does the new auditing scheme change compared with prior work?\",\"answer\":\"It delivers tight privacy estimates for natural (non-adversarial) datasets when the adversary observes all model updates, and it requires only two training runs to achieve tight estimates.\"},{\"question\":\"How does the scheme achieve tighter and more sample-efficient privacy estimation?\",\"answer\":\"It exploits mechanism-specific structure by tailoring the auditing approach to the specific privacy mechanisms used in DP-SGD, adapting techniques for Gaussian DP and functional DP to infer rare-outcome privacy effects from more common observations.\"}]","Tight Auditing of Differentially Private Machine Learning | PDF",1785719977,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"tight-auditing-of-differentially-private-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/tight-auditing-of-differentially-private-machine-learning/118734/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are existing differential-privacy auditing methods considered too limited?","Question",{"text":75,"@type":76},"They yield tight estimates only under implausible worst-case assumptions, and they require thousands or millions of training runs to estimate privacy leakage with non-trivial statistical accuracy.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the new auditing scheme change compared with prior work?",{"text":80,"@type":76},"It delivers tight privacy estimates for natural (non-adversarial) datasets when the adversary observes all model updates, and it requires only two training runs to achieve tight estimates.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the scheme achieve tighter and more sample-efficient privacy estimation?",{"text":84,"@type":76},"It exploits mechanism-specific structure by tailoring the auditing approach to the specific privacy mechanisms used in DP-SGD, adapting techniques for Gaussian DP and functional DP to infer rare-outcome privacy effects from more common observations.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]