Approximate, Adapt, Anonymize (3A) - a Framework for Privacy Preserving Training Data Release for Machine Learning

Sep 17, 2026

Uploaded by Miles

Large informative datasets are crucial for machine learning, yet releasing high-utility data in sensitive domains while protecting individual privacy remains difficult. This paper presents the 3A framework (Approximate, Adapt, Anonymize) that maximizes ML data utility under differential privacy. It learns an approximation of the data distribution, adapts it to preserve downstream model loss metrics, and anonymizes via noise addition for (ε,δ)-DP. A mixture-model implementation with kernel-inducing-point adaptation and Gaussian DP yields minimal performance discrepancy vs. real-data training, outperforming prior privacy-preserving synthetic generators.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Abstract

Introduction

Privacy-preserving synthetic data generation framework maximizing utility for machine learning

Approximate the underlying data distribution

Adapt to preserve classifier loss

Anonymize with differential privacy

Approximate, Adapt, Anonymize (3A): a Framework for Privacy Preserving Training Data Release for Machine Learning Tamas Madl* , Weijie Xu, Olivia Choudhury, Matthew Howard* arXiv :2307 .0 1875v 1 [ cs .LG] 4 Jul 2023 Abstract The availability of large amounts of informative data is crucial for successful machine learning. However, in domains with sensitive information, the release of high-utility data which protects the privacy of individuals has proven challenging. Despite progress in differential privacy and generative modeling for privacy-preserving data release in the literature, only a few approaches optimize for machine learning utility: most approaches only take into account statistical metrics on the data itself and fail to explicitly preserve the loss metrics of machine learning models that are to be subsequently trained on the generated data. In this paper, we introduce a data release framework, 3A (Approximate, Adapt, Anonymize), to maximize data utility for machine learning, while preserving differential privacy. The framework aims to 1) learn an approximation of the underlying data distribution, 2) adapt it such that loss metrics of machine learning models are preserved as closely as possible, and 3) anonymize by using a noise addition mechanism to ensure differential privacy. We also describe a speciﬁc implementation of this framework that leverages mixture models to approximate, kernelinducing points to adapt, and Gaussian differential privacy toanonymize a dataset, in order to ensure that the resulting data is both privacy-preserving and high utility. We present experimental evidence showing minimal discrepancy between performance metrics of models trained on real versus privatized datasets, when evaluated on held-out real data. We also compare our results with several privacy-preserving synthetic data generation models (such as differentially private generative adversarial networks), and report signiﬁcant increases in classiﬁcation performance metrics compared to state-ofthe-art models. These favorable comparisons show that the presented framework is a promising direction of research, increasing the utility of low-risk synthetic data release for machine learning. Introduction Training and using machine learning models in domains with sensitive and personally identiﬁable information such as healthcare presents signiﬁcant legal, ethical and trust challenges; slowing the progress of this technology as well as its potential positive impact. Data synthesis has been proposed as a potential mechanism that can be a legally and *These authors are from AWS. Others are from Amazon. Please [contact mdlt@amazon.nl](contact mdlt@amazon.nl) if you have any question. ethically appropriate solution to the sharing and processing of sensitive data. It is possible to synthesize a dataset that allows accurate machine learning (ML) model training without compromising privacy, depending on privacy deﬁnition and requirements. In the case of the GDPR, for example, data that do not allow singling out may be excepted from the regulation (Cohen and Nissim 2020) . In this work, we will rely on differential privacy (Dwork 2006), which unlike weaker privacy criteria such as k-anonymity has been shown to protect against singling out individuals (Cohen and Nissim 2020) . A privacy-preserving synthetic data generation framework maximizing utility for machine learning For the purposes of privacy-preserving ML by means of differentially private (DP) data release, we are interested in approaches which 1) approximate the true data distribution, 2) preserve utility for machine learning (ML models trained on the data release perform similarly to models trained on true data), and 3) preserve privacy in accordance to DP. More formally, we propose studying a class of data generation algorithms M which, given an original dataset D =(Xi ; Yi) with n~ data points Xi and labels Yi , produce a synthetic dataset D = M (D), such that the

FAQs

What problem does the 3A framework address in privacy-preserving ML data release?

It addresses the challenge of releasing high-utility data in sensitive domains while explicitly preserving the loss metrics of ML models trained on the released data, under differential privacy.

How does 3A preserve utility for machine learning?

It uses a three-step process: approximate the underlying data distribution, adapt the approximation so that a classifier’s loss on sampled data matches the loss on original data, and then anonymize while maintaining DP.

What experimental evidence supports the effectiveness of the proposed framework?

The paper reports minimal discrepancy between performance metrics of models trained on real versus privatized datasets on held-out real data, and it shows significant classification metric improvements over state-of-the-art privacy-preserving synthetic data generation models.

Sep 17, 2026

Uploaded by Miles