SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation - Secure Multi-party Learning and Federated Learning Survey

Aug 3, 2026

Upload by Jiven

Training high-quality machine learning models across multiple data controllers with privacy preservation is a central challenge in the big-data era. Secure Multi-party Learning (MPL) and Federated Learning (FL) are two mainstream solution directions, each with distinct privacy guarantees, communication patterns, overhead, data formats, model accuracy, and deployment scenarios. This survey defines the TMMPP problem, compares recent technical routes, parties, partitioning, threat models, and supported learning models, reviews state-of-the-art platforms for online training, and outlines future directions.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Training with privacy preservation across multiple data controllers

TMMPP overview and two mainstream solution directions

Problem definition and comparative evaluation (MPL vs FL)

Survey scope and motivation

View metadata, citation and similar [papers at ](papers at core.ac.uk)[core.ac.uk](papers at core.ac.uk) brought to you by CORE [provided by](provided by arXiv.org)[ arXiv.org](provided by arXiv.org) e-Print Archive SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation Lushan Song, Haoqi Wu, Wenqiang Ruan, Weili Han Laboratory for Data Analytics and Security, Fudan University arXiv :2012 .03386v 1 [ cs .CR] 6 Dec 2020 Abstract—Nowadays, gathering high-quality training data from multiple data controllers with privacy preservation is a key challenge to train high-quality machine learning models. The potential solutions could dramatically break the barriers among isolated data corpus, and consequently enlarge the range of data available for processing. To this end, both academia researchers and industrial vendors are recently strongly motivated to propose two main-stream folders of solutions: 1) Secure Multi-party Learning (MPL for short); and 2) Federated Learning (FL for short). These two solutions have their advantages and limitations when we evaluate them from privacy preservation, ways of communication, communication overhead, format of data, the accuracy of trained models, and application scenarios. Motivated to demonstrate the research progress and discuss the insights on the future directions, we thoroughly investigate these protocols and frameworks of both MPL and FL. At ﬁrst, we deﬁne the problem of training machine learning models over multiple data sources with privacy-preserving (TMMPP for short). Then, we compare the recent studies of TMMPP from the aspects of the technical routes, parties supported, data partitioning, threat model, and supported machine learning models, to show the advantages and limitations. Next, we introduce the state-of-theart platforms which support online training over multiple data sources. Finally, we discuss the potential directions to resolve the problem of TMMPP. I. INTRODUCTION In the era of big data, almost all online activities are driven by data. As is said by IBM's Chief Executive Ofﬁcer that“big data is the new oil”, the data would bring us huge beneﬁts, therefore have become the key in current business competitions. In addition, these data are pushing the advances of machine learning which have brought us such breakthroughsin various scenarios, such as medical diagnosis [1][2], imageclassiﬁcation [3], and facial recognition [4] . In such aforementioned widely-applied scenarios, massive amounts of highquality data are the dominating factor in the performance of the machine learning models. However, researchers are faced with a tough predicament where the needed data are hard to be collected and shared directly to jointly train high-performance machine learning models, because these data protected by privacy protection policies or regulations might contain much sensitive or private information, and are usually stored in multiple locations. Recently, the released laws, such as General Data Protection Regulation (GDPR) [5] which is presented by the European Union, aggravates this predicament during data sharing. Here, GDPR deﬁnes three key roles in data sharing: the data subject, which refers that “an identiﬁed or identiﬁable natural person who owns the personal data”; the data controller, which refers that “the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data”; and the data processor, which refers that “a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller”. The role considered in this paper is the data controller. How to utilize the decentralized data in multiple data controllers to train high-performance machine learning models efﬁciently with privacy preservation is a key challenge. In this challenging scenario, the data controllers own v

FAQs

What does TMMPP mean in this document?

TMMPP stands for training machine learning models over multiple data sources with privacy-preserving constraints. It is the core problem the survey investigates.

What are the next steps after defining and comparing TMMPP approaches?

The survey introduces state-of-the-art platforms that support online training over multiple data sources, then discusses potential future directions to resolve TMMPP.

Aug 3, 2026

Upload by Jiven