SoK - Let the Privacy Games Begin! - A Unified Treatment of Data Inference Privacy in Machine Learning

Aug 3, 2026

Upload by Logic

Machine learning models deployed in production can enable adversaries to infer sensitive information about training data. The literature studies many inference risks, from membership inference to reconstruction attacks, and some works adopt a game-based style inspired by probabilistic experiments in cryptography. Prior presentations often differ subtly in adversary capabilities and objectives, making results hard to compare and combine. This paper introduces a game-based framework that systematizes privacy inference risks. It unifies risk definitions, proves formal relationships, and reveals previously unknown connections, enabling rigorous composition and clearer understanding of inference privacy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Introduction

Privacy inference risks and privacy games

Framework overview and contributions

Game-based framework (inferred)

Anatomy of game-based privacy definitions

Unified representation of fundamental risks

Relations among risks (inferred)

Formal relationships and reduction/separation notions

Security implications under threat models

SoK: Let the Privacy Games Begin! A Uniﬁed Treatment of Data Inference Privacy in Machine Learning arXiv :2212 . 10986v2 [ cs .LG] 20 Apr 2023 Ahmed Salem􀀃z , Giovanni Cherubin􀀃 , David Evansy , Boris Kpf􀀃 Andrew Paverd􀀃 , Anshuman Suriy , Shruti Tople􀀃 , Santiago Zanella-Bguelin􀀃z 􀀃 Microsoft ft-salem.ahmed, giovanni.cherubin, boris.koepf, andrew.paverd, shruti.tople, [santiago](santiagog@microsoft.com)[g](santiagog@microsoft.com)[@microsoft.com](santiagog@microsoft.com) y University of Virginia fevans, [as9rw](as9rwg@virginia.edu)[g](as9rwg@virginia.edu)[@virginia.edu](as9rwg@virginia.edu) Abstract—Deploying machine learning models in production may allow adversaries to infer sensitive information about training data. There is a vast literature analyzing different types of inference risks, ranging from membership inference to reconstruction attacks. Inspired by the success of games (i.e. probabilistic experiments) to study security properties in cryptography, some authors describe privacy inference risks in machine learning using a similar game-based style. However, adversary capabilities and goals are often stated in subtly different ways from one presentation to the other, which makes it hard to relate and compose results. In this paper, we present a game-based framework to systematize the body of knowledge on privacy inference risks in machine learning. We use this framework to (1) provide a unifying structure for deﬁnitions of inference risks, (2) formally establish known relations among deﬁnitions, and (3) to uncover hitherto unknown relations that would have been difﬁcult to spot otherwise. Index Terms—privacy, machine learning, differential privacy, membership inference, attribute inference, property inference I. INTRODUCTION Since the pioneering studies of attribute inference [22, 69] and membership inference [37, 56], research on the inference risks of deploying machine learning (ML) models has bloomed. There is a growing interest in understanding and mitigating the leakage of information about training data under various threat models that capture different adversarial capabilities (e.g., observing model outputs, model parameters, or transcripts of iterative optimization methods) and goals (e.g., membership inference [56], attribute inference [22, 69], property inference [23, 42, 58, 74], and data reconstruction [4, 11]) . An emerging trend in the literature is to capture threat models using privacy games. This originates from the seminal work of Wu et al. [69] on formalizing attribute inference. A privacy game is a probabilistic experiment where an adversary interacts with a challenger. The challenger drives the experiment, invoking the adversary to provide them with information and to allow them to make certain choices, possibly while interacting with oracles controlled by the challenger. The adversary eventually produces a guess for a conﬁdential value. z Corresponding author This experiment deﬁnes a probability space where the success of the adversary can be measured in terms of the probability of their guess being correct. The use of games for privacy in ML is inspired by the wellestablished use of games to deﬁne and reason about security properties in cryptography. Cryptographic games are used to standardize and compare security deﬁnitions [25, 57], and to structure [6] and even mechanize proofs of security [5, 9] . In comparison, the use of privacy games in the ML literature is still in its infancy: (1) there are no well-established standards for game-based deﬁnitions, (2) relationships between different privacy games have only been partially explored, and (3) games are rarely used as an integral part of proofs, despite being especially convenient for this task. This has resulted in many game variants in the literature that attempt to formalize the same adversary goal but have subtle yet important differences. This fragmentation leads to confusion and hinders progress—for

FAQs

Which privacy risks are represented and connected as games in the paper?

The framework represents five fundamental risks as games: membership inference, attribute inference, property inference, differential privacy distinguishability, and data reconstruction, and then proves rigorous relations among them.

Aug 3, 2026

Upload by Logic