Robust Representation Learning for Privacy-Preserving Machine Learning - A Multi-Objective Autoencoder Approach

Aug 4, 2026

Upload by Anda

Privacy-preserving machine learning addresses growing legal and ethical requirements driven by heavy reliance on data. Existing approaches either rely on cryptography such as homomorphic encryption or add noise via differential privacy, often causing significant performance degradation and practical complexity. This work proposes deep robust representation learning by training autoencoders in a multi-objective manner and concatenating latent and learned encoder features as the shared representation. The encoded data can be sent to third parties for intensive training and hyperparameter tuning while reducing risks of revealing original inputs. Experiments on unimodal and multimodal vertical splitting settings show improved performance over state of the art.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

Privacy threats in machine learning

Limitations of cryptography-based methods

Limitations of differential privacy

Proposed approach

Multi-objective residual autoencoder and feature encoding

Threat analysis and adversary access models

Experimental validation

Unimodal and multimodal (vertical splitting) settings

Comparison with state of the art

Robust Representation Learning for Privacy-Preserving Machine Learning: A Multi-Objective Autoencoder Approach Sofiane Ouaari 1 ,2 , Ali Burak Ünal 1 ,2 ,3 , Mete Akgün 1 ,2 ,3 and Nico Pfeifer1 ,2 1Methods in Medical Informatics, Department of Computer Science, University of Tuebingen, Germany 2Institute for Bioinformatics and Medical Informatics (IBMI), University of Tuebingen, Germany 3Medical Data Privacy and Privacy Preserving Machine Learning, University of Tuebingen, Germany {sofiane.ouaari, ali-burak.uenal, mete.akguen, nico.pfeifer}@uni-tuebingen.de arXiv :2309 .04427v 1 [ cs .LG] 8 Sep 2023 1 Abstract—Several domains increasingly rely on machine learning in their applications. The resulting heavy dependence on data has led to the emergence of various laws and regulations around data ethics and privacy and growing awareness of the need for privacy-preserving machine learning (ppML). Current ppML techniques utilize methods that are either purely based on cryptography, such as homomorphic encryption, or that introduce noise into the input, such as differential privacy. The main criticism given to those techniques is the fact that they either are too slow or they trade off a model’s performance for improved confidentiality. To address this performance reduction, we aim to leverage robust representation learning as a way of encoding our data while optimising the privacy-utility trade-off. Our method centers on training autoencoders in a multi-objective manner and then concatenating the latent and learned features from the encoding part as the encoded form of our data. Such a deep learning-powered encoding can then safely be sent to a third party for intensive training and hyperparameter tuning. With our proposed framework, we can share our data and use third party tools without being under the threat of revealing its original form. We empirically validate our results on unimodal and multimodal settings, the latter following a vertical splitting system and show improved performance over state-of-the-art. I. INTRODUCTION A wide range of application sectors is drastically integrating machine learning (ML) in diverse products. A successful ML model often requires a huge amount of training data and powerful computational resources. However, the need for such enormous volumes of data to develop performing models raises serious privacy concerns. Such ML models might face multiple types of adversarial attacks depending on the type of access an adversary might have to the model (white or black-box) . A membership inference attack (Shokri et al., 2017) allows an attacker to query a trained machine learning model to predict whether a given example is in the model’s training data set. On the other hand, an inversion attack (Fredrikson et al., 2015; K.-C. Wang et al., 2021; Ye et al., 2022) aims to recreate an input data point given a confidence score obtained from a black-box inference of the model. In order to make researchers and engineers take such privacy threats into consideration, many regulations and ethical data policies, such as GDPR, 1Preprint. Under review CCPA, and CPRA (Hijmans & Raab, 2018; Rochel, 2021) were set to raise awareness around this topic and restrict any data violations that might occur in a given ML pipeline. Previous works have been done to reduce the effectiveness of different privacy attacks. Among those studies, differential privacy (DP) is the most commonly used approach which operates by incorporating predetermined randomization into a machine learning algorithm’s computation. The perturbation introduced by DP might be applied on the users’s input, parameters, prediction output and even on loss functions (Abadiet al., 2016; Phan et al., 2016) . However, many studies have shown that such noise reduces the performance of the model for the sake of privacy (Truex et al., 2019) . Furthermore, Setting up a correct value for ϵ is complex by nature and requires some trial and error process

FAQs

为什么隐私保护机器学习需要关注隐私与效用的权衡？

依赖数据训练的机器学习模型会引入隐私风险，而常见隐私技术往往要么带来较高计算开销，要么降低模型性能，因此需要在隐私与效用之间取得平衡。

与同态加密和差分隐私相比，文中方法的核心思路是什么？

文中通过多目标训练的自动编码器对数据进行深度表示学习，将潜在与编码端学习到的特征拼接成可共享的表示，从而在分享给第三方训练与调参时降低泄露原始数据的威胁。

文中如何验证所提框架的有效性？

通过在单模态与多模态（采用垂直划分机制）两类设置下进行实证评估，并与现有最先进方法进行比较，结果表明性能有所提升。

Aug 4, 2026

Upload by Anda