[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122776-en":3,"doc-seo-122776-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122776,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Estimating Privacy Leakage of Machine Learning Models - Thesis Abstract","Membership inference attacks extract information about training data from machine learning models. This thesis extends prior worst-case vulnerability analysis, replacing the Bayes optimal classifier with practical estimators for a computable subclass of membership inference problems. Extensive simulations on real-world datasets show that costly privacy auditing methods such as shadow modeling can be replaced by the proposed estimators. The work also quantifies disparity, demonstrating that membership inference risk varies across population subgroups, and argues that average attack accuracy metrics may not reflect these differences.","UCLA  \nUCLA Electronic Theses and Dissertations  \nTitle  \nEstimating Privacy Leakage of Machine Learning Models  \nPermalink  \n[https://escholarship.org/uc/item/1ss8n3ks](https://escholarship.org/uc/item/1ss8n3ks)  \nAuthor  \nO'Dell, Ryan  \nPublication Date  \n2023  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nUNIVERSITY OF CALIFORNIA Los Angeles  \nEstimating Privacy Leakage of Machine Learning Models  \nA thesis submitted in partial satisfaction of the requirements for the degree Master of Science in Statistics  \nby  \nRyan O’Dell  \n2023  \nABSTRACT OF THE THESIS  \nEstimating Privacy Leakage  \nof Machine Learning Models  \nby  \nRyan O’Dell  \nMaster of Science in Statistics  \nUniversity of California, Los Angeles, 2023  \nProfessor Guang Cheng, Chair  \nA membership inference attack is a method of extracting the training data from machine learning models. Previous analysis has characterized the worst case vulnerability to membership inference by instantiating the attack algorithm as the Bayes Optimal Classifier. We extend these findings by developing practical estimators for the worst case vulnerability on a sub-class of membership inference problems that are easy to compute without resorting to computationally expensive privacy auditing techniques. Extensive simulation studies are conducted on real world data sets to show that privacy auditing techniques, such as shadow modeling, can be replaced with the proposed worst case estimators. Furthermore, we examine the notion of disparity in membership inference: that some subgroups of the population are easier to identify in the training data set than others. We use a framework to quantify the degree of disparity and demonstrate that several real world models exhibit disparity in membership inference. We advocate that average metrics of attack accuracy, commonly used in the privacy auditing literature, do not reliably convey the difference in privacy risks across different levels of the population.  \nThe thesis of Ryan O’Dell is approved.  \nGuido Francisco Mont´ufar Cuartas  \nMark S. Handcock  \nGuang Cheng, Committee Chair  \nUniversity of California, Los Angeles 2023  \nTo my family    \nand all those who supported me during my academic journey.  \niv  \nTABLE OF CONTENTS  \n1 Introduction ...................................... 1  \n2 Machine Learning .................................. 3  \n2.1 Regression ..................................... 4  \n2.2 Classification ................................... 5  \n2.3 Stochastic Gradient Descent ........................... 6  \n3 Di􀀋erential Privacy ................................. 9  \n3.1 ǫ-Differential Privacy ............................... 10  \n3.2 (ǫ,δ)-Differential Privacy ............................. 13  \n4 Membership Inference ............................... 16  \n4.1 Formalizing Membership Inference ........................ 17  \n4.2 Methods of Membership Inference ........................ 19  \n4.2.1 Metric Based Attack ........................... 20  \n4.2.2 Shadow Modeling Attack ......................... 21  \n5 Characterizing Membership Inference ...................... 24  \n5.1 Worst Case Vulnerability ............................. 24  \n5.2 Distributional Generalization .......................... 25  \n5.3 Estimating Vulnerability ............................. 28  \n5.4 Estimating Worst Case Vulnerability ...................... 31  \n6 Disparity ........................................ 33  \n6.1 Disparity ...................................... 33  \n6.2 Estimating Disparity ............................... 34  \n7 Experiments ...................................... 36  \n7.1 Law School Data ................................. 36  \n7.2 Adult Data .................................... 38  \n7.3 Experimental Design ............................... 39  \n7.3.1 Victim Model Implementation ...................... 39  \n7.3.2 Attack Model Implementation ...................... 40","cbCaibCXqirockgZ","https://ap.wps.com/l/cbCaibCXqirockgZ","pdf",419417,1,63,"English","en",105,"# Introduction\n# Machine Learning\n## Regression\n## Classification\n## Stochastic Gradient Descent\n# Differential Privacy\n## ǫ-Differential Privacy\n## (ǫ,δ)-Differential Privacy\n# Membership Inference\n## Formalizing Membership Inference\n## Methods of Membership Inference\n# Characterizing Membership Inference\n## Worst Case Vulnerability\n## Estimating Vulnerability\n# Disparity\n## Estimating Disparity\n# Experiments\n## Law School Data\n## Adult Data\n# Discussion\n## Practitioners Considerations\n## Future Directions","[{\"question\":\"What does membership inference mean in the thesis?\",\"answer\":\"It refers to attacks that extract training-data membership information from machine learning models.\"},{\"question\":\"How does the thesis extend worst-case vulnerability analysis?\",\"answer\":\"It develops practical estimators for worst-case vulnerability on a computationally efficient subclass of membership inference problems, avoiding expensive privacy auditing.\"},{\"question\":\"Why is disparity important for privacy risk evaluation?\",\"answer\":\"The thesis shows that some population subgroups are easier to identify in training data, and quantifies disparity to demonstrate that privacy risk can differ across levels of the population.\"}]","Estimating Privacy Leakage of Machine Learning Models - Thesis Abstract | PDF",1785812834,159,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"estimating-privacy-leakage-of-machine-learning-models-thesis-abstract","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/estimating-privacy-leakage-of-machine-learning-models-thesis-abstract/122776/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does membership inference mean in the thesis?","Question",{"text":75,"@type":76},"It refers to attacks that extract training-data membership information from machine learning models.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis extend worst-case vulnerability analysis?",{"text":80,"@type":76},"It develops practical estimators for worst-case vulnerability on a computationally efficient subclass of membership inference problems, avoiding expensive privacy auditing.",{"name":82,"@type":73,"acceptedAnswer":83},"Why is disparity important for privacy risk evaluation?",{"text":84,"@type":76},"The thesis shows that some population subgroups are easier to identify in training data, and quantifies disparity to demonstrate that privacy risk can differ across levels of the population.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]