On the Robustness of Distributed Machine Learning against Transfer Attacks - arXiv 2412.14080v1

Aug 5, 2026

Upload by Ophelia

Distributed machine learning attracts wide interest, yet existing studies analyze robustness in distributed settings by isolating either training or inference. This work investigates the combined robustness achieved when both learning and inference are distributed, including fully heterogeneous configurations across training data, architecture, scheduler, optimizer, and other parameters. Theory and extensive CIFAR10 and FashionMNIST experiments show consistent improvements in the accuracy–robustness tradeoff against strong transfer-based attacks. On CIFAR10, for the Common Weakness attack, robust accuracy increases by up to 40% with minimal impact on clean accuracy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

Background and Motivation

Research Questions

Robustness Evaluation

On the Robustness of Distributed Machine Learning against Transfer Attacks Sébastien Andreina 1 , Pascal Zimmer2 , Ghassan Karame2 1 NEC Labs Europe, Germany 2 Ruhr University Bochum, Germany [sebastien.andreina@neclab.eu](sebastien.andreina@neclab.eu), {pascal.zimmer, [ghassan.karame}@rub.de](ghassan.karame}@rub.de) arXiv :2412 . 14080v1 [ cs .LG] 18 Dec 2024 Abstract Although distributed machine learning (distributed ML) is gaining considerable attention in the community, prior works have independently looked at instances of distributed ML in either the training or the inference phase. No prior work has examined the combined robustness stemming from distributing both the learning and the inference process. In this work, we explore, for the first time, the robustness of distributed ML models that are fully heterogeneous in training data, architecture, scheduler, optimizer, and other model parameters. Supported by theory and extensive experimental validation using CIFAR10 and FashionMNIST, we show that such properly distributed ML instantiations achieve across-the-board improvements in accuracy-robustness tradeoffs against state-of-the-art transfer-based attacks that could otherwise not be realized by current ensemble or federated learning instantiations. For instance, our experiments on CIFAR10 show that for the Common Weakness attack, one of the most powerful state-of-the-art transfer-based attacks, our method improves robust accuracy by up to 40%, with a minimal impact on clean task accuracy. 1 1 Introduction Nowadays, many applications increasingly rely on various forms of distributed learning. For instance, Google spell-checking utilizes horizontal federated learning (Zhang et al. 2023b), and Apple, among others, may also be adopting similar approaches (Paulik et al. 2021) . Autonomous vehicles are soon expected to follow suit (Chellapandi et al. 2024) . In the financial sector, vertical federated learning is currently employed (Liu et al. 2024), and many researchers have already begun exploring decentralized learning without a central authority (Lian et al. 2017; Dhasade et al. 2023) . Although federated and decentralized learning is gaining considerable interest and attention from practitioners and researchers, the concept of distributed learning was first attempted with ensemble learning. Here, weak learners employing different model architectures (typically co-located on the same machine) jointly train 1 The code is available at [https://github](https://github.com/RUB)[.](https://github.com/RUB)[com/RUB](https://github.com/RUB)InfSec/distributed_learning_robustness. using the same dataset; inference requires some sort of majority vote among those weak learners. Unlike ensemble learning, federated learning typically requires weak learners to jointly train a global model (optionally with the help of a central server) . The main motivation of these instantiations is to obviate the need for weak learners to share their local training data, hence increasing learners’ privacy while allowing for diverse data to be effectively used for training. As such, current federated learning approaches primarily aim to diversify data sources for training but do not directly alter the inference process. In contrast, ensemble learning focuses mainly on inference by enhancing model diversity. The literature features a number of contributions that analyze the privacy provisions of FL and its resistance to backdoors on the one hand and the robustness offered by ensemble learning on the other hand. No prior work has examined the combined robustness and security pro visions stemming from distributing both the learning and the inference process . This is particularly relevant since centralized ML models, such as deep neural networks, have been shown to lack robustness against adversarial examples (Szegedy et al. 2014; Biggio et al. 2013) . In this paper, we address this gap and explore whether pure distributed lear

FAQs

What gap does the paper address about robustness in distributed machine learning?

Prior work examined robustness by looking separately at distributed learning instances during either training or inference. The paper studies robustness that arises when both learning and inference are distributed together.

What makes the proposed distributed ML setting different from ensemble or federated learning?

The setting is a hybrid that keeps benefits of federated/decentralized learning and traditional ensembles, allowing learners to independently choose both training parameters (e.g., optimizer and scheduler) and model architecture parameters, in addition to training data.

How does the method perform against transfer-based attacks on CIFAR10?

Experiments on CIFAR10 show that for the Common Weakness attack, robust accuracy improves by up to 40% while causing minimal impact on clean-task accuracy.

Aug 5, 2026

Upload by Ophelia