Balancing Privacy, Robustness, and Efficiency in Machine Learning - Paper - Issue 2312

Aug 4, 2026

Upload by Maeve

Position paper argues that achieving robustness, user privacy, and system-wide efficiency at the same time in machine learning is infeasible under conventional worst-case adversarial threat models. The paper attributes the tension to structural limitations from adversarial assumptions rather than to specific algorithmic gaps. It proposes formalizing a robustness–privacy–efficiency trilemma, studying how relaxing threat models can enable better trade-offs, and creating benchmarks that reveal compromises instead of hiding them. It also calls for context-aware deployment design and transparent reporting along all three axes, especially as foundation models and regulation increase practical stakes.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Introduction

Limitations of worst-case models

Why now?

Contributions

Related work

arXiv :2312 . 14712v3 [ cs .LG] 26 Jun 2025 Balancing Privacy, Robustness, and Efficiency in Machine Learning Youssef Allouah Rachid Guerraoui John Stephan EPFL, Switzerland Abstract This position paper argues that achieving robustness, privacy, and efficiency simultaneously in machine learning systems is infeasible under prevailing threat models. The tension between these goals arises not from algorithmic shortcomings but from structural limitations imposed by worst-case adversarial assumptions. We advocate for a systematic research agenda aimed at formalizing the robustness-privacyefficiency trilemma, exploring how principled relaxations of threat models can unlock better trade-offs, and designing benchmarks that expose rather than obscure the compromises made. By shifting focus from aspirational universal guarantees to context-aware system design, the machine learning community can build models that are truly appropriate for real-world deployment. 1 Introduction Machine learning systems now underpin high-stakes applications, such as cancer diagnosis in radiology, legal document automation, and predictive keyboards used daily by billions of people, dramatically increasing the stakes of privacy violations and adversarial manipulations. Each deployment is expected to protect user data, resist malicious manipulation, and run at large scale. Meeting all three demands simultaneously has proven elusive. Wan et al. [77] showed that as few as 100 poisoned instruction-tuning examples can reliably subvert GPT-style models across hundreds of tasks. On the edge-device side, Bagdasaryan et al. [9] demonstrated that only two compromised phones can implant a hidden back-door in a federated keyboard predictor equipped with secure aggregation and client-side differential privacy; Suliman and Leith [74] later reproduced the attack against Google GBoard at production scale. These findings suggest a broader structural tension: essentially, strengthening one pillar of robustness, privacy, or efficiency often compromises the remaining two. Limitations of worst-case models. Many existing analyses default to the strongest adversary assumptions: fully malicious clients, local differential privacy without any trusted intermediary, and full collusion among attackers. While these worst-case models yield clean theoretical guarantees, they often overshoot practical needs and obscure realistic trade-offs. In many applications, adversaries have limited coordination, semi-honest data shufflers exist, or partial trust anchors can be leveraged. Recognizing where worst-case assumptions are unnecessarily pessimistic is key to designing systems that recover valuable efficiency or robustness without materially raising actual risk. Position: Under conventional worst-case threat models (adversaries), a machine learning system cannot simultaneously satisfy robustness to corruption, user-level privacy, and system-wide efficiency. Rather than pursuing an unattainable ideal, we advocate explicit and context-dependent compromises with transparent reporting on all three axes. Why now? Foundation models amplify the stakes: a single poisoned prompt, once assimilated during fine-tuning, can ripple across thousands of downstream applications [77]; conversely, stronger integrity checks can multiply training cost by orders of magnitude. At the same time, rapidly advancing regulations—such as Europe’s AI Act [32] which explicitly mandates transparency in robustness and privacy trade-offs, and increasingly stringent healthcare and financial-sector regulations requiring demonstrable guarantees—are pushing these tensions urgently into practical focus. Practitioners sometimes ignore the tension, ending up with solutions that do not scale, are not robust or not private. Sometimes they are forced to choose which requirement to relax, often without guidance on the systemic implications. A principled framework for reasoning about this three-way trade-off is urgent

FAQs

Why does the paper claim robustness, privacy, and efficiency cannot all be achieved simultaneously?

Under conventional worst-case threat models, structural limitations create unavoidable trade-offs, so meeting all three pillars together is infeasible rather than just difficult.

What does the paper mean by “worst-case models” and why are they a problem?

They assume fully malicious clients, local differential privacy without trusted intermediaries, and full collusion among attackers, which can be overly pessimistic and obscure realistic compromises.

What research agenda does the paper advocate?

It recommends formalizing the robustness–privacy–efficiency trilemma, exploring relaxations of threat models to improve trade-offs, and designing benchmarks that explicitly expose the compromises in practice.

Aug 4, 2026

Upload by Maeve