[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117894-en":3,"doc-seo-117894-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117894,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Information Leakage from Data Updates in Machine Learning Models - Research Report","Machine learning models are often retrained to incorporate updated datasets, reflect distribution shifts, and support requirements such as data deletion. This work studies whether an adversary can infer information about changed training attributes when the attacker has snapshots of the model before and after the update. The paper introduces attacks exploiting differences in prediction confidence, evaluates them on public datasets with standard classifiers, and shows snapshot access increases leakage. It further finds that rare attribute values are more vulnerable and repeated updates amplify the attacker’s ability to guess updated values.","Information Leakage from Data Updates in Machine Learning  \nModels  \nTian Hui The University of Melbourne  \nAustralia [huith@student.unimelb.edu.au](huith@student.unimelb.edu.au)  \nFarhad Farokhi  \nThe University of Melbourne Australia [ffarokhi@unimelb.edu.au](ffarokhi@unimelb.edu.au)  \nOlga Ohrimenko The University of Melbourne  \nAustralia [oohrimenko@unimelb.edu.au](oohrimenko@unimelb.edu.au)  \narXiv :2309 . 11022v1 [ cs .LG] 20 Sep 2023  \nABSTRACT  \nIn this paper we consider the setting where machine learning models are retrained on updated datasets in order to incorporate the most up-to-date information or reflect distribution shifts. We investigate whether one can infer information about these updatesin the training data (e.g., changes to attribute values of records) . Here, the adversary has access to snapshots of the machine learning model before and after the change in the dataset occurs. Contrary to the existing literature, we assume that an attribute of a single or multiple training data points are changed rather than entire data records are removed or added. We propose attacks based on the difference in the prediction confidence of the original model and the updated model. We evaluate our attack methods on two public datasets along with multi-layer perceptron and logistic regression models. We validate that two snapshots of the model can result in higher information leakage in comparison to having access to only the updated model. Moreover, we observe that data records with rare values are more vulnerable to attacks, which points to the disparate vulnerability of privacy attacks in the update setting. When multiple records with the same original attribute value are updated to the same new value (i.e., repeated changes), the attacker is more likely to correctly guess the updated values since repeated changes leave a larger footprint on the trained model. These observations point to vulnerability of machine learning models to attribute inference attacks in the update setting.  \nCCS CONCEPTS  \n• Security and privacy; • Computing methodologies → Machine learning;  \nKEYWORDS  \nMachine Learning; Privacy; Attribute Inference; Data Update.  \nACM Reference Format:  \nTian Hui, Farhad Farokhi, and Olga Ohrimenko. 2023. Information Leakage from Data Updates in Machine Learning Models. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec ’23), November 30, 2023, Copenhagen, Denmark. ACM, New York, NY, USA, 7 pages. [https:](https:)//[doi.org/10.1145/3605764.3623905](doi.org/10.1145/3605764.3623905)  \nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission [and/or a fee. Request permissions from permissions@acm.org](and/or a fee. Request permissions from permissions@acm.org).  \nAISec ’23, November 30, 2023, Copenhagen, Denmark  \n© 2023 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN 979-8-4007-0260-0/23/11. . . $15.00  \n[https://doi.org/10.1145/3605764.3623905](https://doi.org/10.1145/3605764.3623905)  \n1 INTRODUCTION  \nMachine learning models are shown to leak private information, such as membership in a training dataset [14], which can be troubling in sensitive application domains. Machine learning models are however not static. Datasets used for training are constantly changing and the model must remain accurate to reflect new trends in data. Therefore, in practice, machine learning models are updated with the arrival of new data to increase accuracy or incorporate distribution shifts in data. F","cbCaioL312DjKoZn","https://ap.wps.com/l/cbCaioL312DjKoZn","pdf",658708,1,7,"English","en",105,"# Abstract\n# Introduction\n## Problem Setting: Updates to Existing Records\n## Attacker Objectives and Assumptions\n# Contributions","[{\"question\":\"What data-update scenario does the paper focus on?\",\"answer\":\"It focuses on updates to existing training records, where attribute values of one or more data points change, rather than adding or removing entire records.\"},{\"question\":\"How do the proposed attacks infer information from model updates?\",\"answer\":\"The attacks use the difference in prediction confidence between the model snapshot before and after the dataset change, enabling attribute inference and record-updated identification.\"},{\"question\":\"What does the paper conclude about snapshot access versus only the updated model?\",\"answer\":\"Access to both the original and updated model snapshots leads to higher information leakage than having access only to the updated model.\"}]","Information Leakage from Data Updates in Machine Learning Models - Research Report | PDF",1785680216,18,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"information-leakage-from-data-updates-in-machine-learning-models-research-report","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/information-leakage-from-data-updates-in-machine-learning-models-research-report/117894/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What data-update scenario does the paper focus on?","Question",{"text":75,"@type":76},"It focuses on updates to existing training records, where attribute values of one or more data points change, rather than adding or removing entire records.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the proposed attacks infer information from model updates?",{"text":80,"@type":76},"The attacks use the difference in prediction confidence between the model snapshot before and after the dataset change, enabling attribute inference and record-updated identification.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the paper conclude about snapshot access versus only the updated model?",{"text":84,"@type":76},"Access to both the original and updated model snapshots leads to higher information leakage than having access only to the updated model.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]