[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118758-en":3,"doc-seo-118758-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118758,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1784791008015729253",8,"Research & Report","DPMLBench - Holistic Evaluation of Differentially Private Machine Learning","Differential privacy formalizes privacy leakage through a privacy budget and, when combined with machine learning, enables differentially private machine learning (DPML) with provable protection. Yet the classic DP-SGD algorithm often suffers substantial utility loss, limiting practical deployment. This work performs a holistic evaluation of improved DPML algorithms, covering utility, defense against membership inference attacks on image classification, and generalizability, spanning multiple algorithms, architectures, datasets, attack settings, and privacy budgets. Results show DP effectively defends against MIAs, while sensitivity-bounding such as per-sample gradient clipping is crucial, and label differential privacy reduces utility loss but can be fragile under MIAs. A modular benchmark software, DPMLBench, supports reproducible comparisons.","To appear in the 30th ACM SIGSAC Conference on Computer and Communications Security. November 26-30, 2023 .  \nDPMLBench: Holistic Evaluation of Differentially Private  \nMachine Learning  \nChengkun Wei 1 Minghu Zhao 1 Zhikun Zhang2 * Min Chen3 Wenlong Meng 1 Bo Liu4 Yuan Fan 1 Wenzhi Chen 1 􀀃  \n1Zhejiang University 2 Stanford University  \n3 CISPA Helmholtz Center for Information Security 4DBAPPSecurity  \narXiv :2305 .05900v1 [ cs .LG] 10 May 2023  \nAbstract  \nDifferential privacy (DP), as a rigorous mathematical definition quantifying privacy leakage, has become a wellaccepted standard for privacy protection. Combined with powerful machine learning techniques, differentially private machine learning (DPML) is increasingly important. As the most classic DPML algorithm, DP-SGD incurs a significant loss of utility, which hinders DPML's deployment in practice. Many studies have recently proposed improved algorithms based on DP-SGD to mitigate utility loss. However, these studies are isolated and cannot comprehensively measure the performance of improvements proposed in algorithms. More importantly, there is a lack of comprehensive research to compare improvements in these DPML algorithms across utility, defensive capabilities, and generalizability.  \nWe fill this gap by performing a holistic measurement of improved DPML algorithms on utility and defense capability against membership inference attacks (MIAs) on imageclassification tasks. We first present a taxonomy of where improvements are located in the machine learning life cycle. Based on our taxonomy, we jointly perform an extensive measurement study of the improved DPML algorithms, over twelve algorithms, four model architectures, four datasets, two attacks, and various privacy budget configurations. We also cover state-of-the-art label differential privacy (Label DP) algorithms in the evaluation. According to our empirical results, DP can effectively defend against MIAs, and sensitivity-bounding techniques such as per-sample gradient clipping play an important role in defense. We also explore some improvements that can maintain model utility and defend against MIAs more effectively. Experiments show that Label DP algorithms achieve less utility loss but are fragile to MIAs. Machine learning practitioners may benefit from these evaluations to select appropriate algorithms. To support our evaluation, we implement a modular re-usable software, DPMLBench, 1 which enables sensitive data owners to deploy DPML algorithms and serves as a benchmark tool for researchers and practitioners.  \n*Corresponding authors.  \n1 The implementation can be found at [https://github.com/](https://github.com/)[ ](https://github.com/)DmsKinson/DPMLBench  \n1 Introduction  \nAs machine learning (ML) continues to evolve, numerous fields are leveraging its power to advance their development [1, 2]; however, this often involves the use of private data, such as medical records. Previous studies have revealed that the models trained on private data can leak information through a bunch of attacks, such as membership inference [3], model inversion [4], and attribute inference [5], which raises critical privacy and security concerns.  \nDifferential privacy (DP) is a widely used notion to rigorously formalize and measure the privacy guarantee based on a parameter called privacy budget. Abadi et al. [6] proposed a general DPML algorithm called differentially private stochastic gradient descent (DP-SGD) by integrating per-sample clipping and noise perturbation to the aggregated gradient in the training process. However, models trained by DP-SGD normally perform badly with respect to model utility. Recently, researchers proposed many improved algorithms with better privacy-utility trade-off [7, 8, 9, 10, 11, 12, 13, 14, 15] . In the rest of this paper, we refer to DP-SGDas vanilla DP-SGD to distinguish between DP-SGD and the improved algorithms.  \nThe improved algorithms modify the vanilla DP-SGD from different aspects but ","cbCaifcqgi37LU68","https://ap.wps.com/l/cbCaifcqgi37LU68","pdf",4518012,1,23,"English","en",105,"# Abstract\n# Introduction\n## Our Contributions\n# References","[{\"question\":\"Why is DP-SGD considered difficult to deploy in practice?\",\"answer\":\"DP-SGD typically incurs significant utility loss, making trained models less accurate. This utility degradation hinders practical deployment of DPML systems.\"},{\"question\":\"What does the DPMLBench holistic evaluation measure?\",\"answer\":\"It jointly evaluates improved DPML algorithms in terms of model utility, defensive capability against membership inference attacks, and generalizability across multiple algorithms, architectures, datasets, attacks, and privacy budgets.\"},{\"question\":\"Which defenses and algorithm improvements are most important for membership inference resistance?\",\"answer\":\"Experiments indicate that differential privacy can effectively defend against MIAs, and sensitivity-bounding techniques such as per-sample gradient clipping play an important role. Some improvements can both maintain utility and improve MIA defense.\"}]","DPMLBench - Holistic Evaluation of Differentially Private Machine Learning | PDF",1785720079,58,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"dpmlbench-holistic-evaluation-of-differentially-private-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/dpmlbench-holistic-evaluation-of-differentially-private-machine-learning/118758/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is DP-SGD considered difficult to deploy in practice?","Question",{"text":75,"@type":76},"DP-SGD typically incurs significant utility loss, making trained models less accurate. This utility degradation hinders practical deployment of DPML systems.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the DPMLBench holistic evaluation measure?",{"text":80,"@type":76},"It jointly evaluates improved DPML algorithms in terms of model utility, defensive capability against membership inference attacks, and generalizability across multiple algorithms, architectures, datasets, attacks, and privacy budgets.",{"name":82,"@type":73,"acceptedAnswer":83},"Which defenses and algorithm improvements are most important for membership inference resistance?",{"text":84,"@type":76},"Experiments indicate that differential privacy can effectively defend against MIAs, and sensitivity-bounding techniques such as per-sample gradient clipping play an important role. Some improvements can both maintain utility and improve MIA defense.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]