Measuring Equality in Machine Learning Security Defenses - Case Study in Speech Recognition

Aug 4, 2026

Upload by Himbo

Over the past decade, machine learning security research has produced many defenses against evasion attacks, yet it often leaves unclear who benefits from these protections. This work connects security defenses with fairness parity metrics to evaluate performance inequities across sub-populations. Using empirical case study results, it shows that common defenses can cause harms such as false rejection and unequal benefits from robustness training. The proposed framework supports multiple defense types and identifies dataset sets for user-centered measurement, including speech recognition results across gender, accent, and age.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

User-centric defense evaluation

Parity metrics and disparate impact

Case study motivation and safety context

Metrics for robustness training and rejection defenses

Measuring Equality in Machine Learning Security Defenses Luke E. Richardsa, b; * , Edward Raffa, c and Cynthia Matuszeka aUniversity of Maryland, Baltimore County b Pacific Northwest National Laboratory cBooz Allen Hamilton arXiv :2302 .08973v 5 [ cs .LG] 1 Jun 2023 Abstract. Over the past decade, the machine learning security community has developed myriad defenses for evasion attacks. An understudied question in that community is: for whom do these defenses defend? This work considers common approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations. We outline appropriate parity metrics for analysis and begin to answer this question through empirical results of the fairness implications of machine learning security methods. We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training. The framework we propose for measuring defense equality can be applied to robustly trained models, preprocessing-based defenses, and rejection methods. We identify a set of datasets with a user-centered application and a reasonable computational cost suitable for case studies in measuring the equality of defenses. In our case study of speech command recognition, we show how such adversarial training and augmentation have non-equal but complex protections for social subgroups across gender, accent, and age in relation to user coverage. We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups. This represents the first work examining the disparity in the adversarial robustness in the speech domain and the fairness evaluation of rejection-based defenses. 1 Introduction Systems integrating machine learning (ML) introduce a new attack surface for adversaries to take advantage of regarding security. So far, we observe that when developing defenses for these systems, only a few works take any metric beyond the increase in robustness to attack account at a high level. However, these systems are rarely considered when designed to interact with humans. In a field where many defenses are already not adequately evaluated, leading to a false sense of security [6], such user-centric evaluations should also be at the forefront. In this work, we expand on the measurement of security ina user-centric manner. We do this by examining the broader set of defenses being introduced in machine learning systems. These include defenses that exist outside of the model weights themselves, like preprocessing and postprocessing. We integrate concepts of parity studied by the machine learning fairness community which measures the equality of performance or outcomes resulting from such a system. Through ∗ Corresponding Author. Email: [lerichards@umbc.edu](lerichards@umbc.edu). this intersectional view, we seek to aid the understanding of the questions the community should ask when integrating various defense defenses. Broadly, this question presents itself: who do these proposed defenses work for when the system is under attack and when not? This comes at a time when machine learning security is being recognized and reaching the maturity of deployment in the private and public sectors. While there exist many complex threat models and levels of access in the literature, we examine impact-based attacks where a defender would attempt to protect the system from potential adversarial inputs. This type of attack represents a level of access only at test time rather than the development of the model. Commonly, the success criteria of an introduced defense are high performance with and without the presence of an attack. In this work, we expand this success criterion to ensure the defenses work for many different user demographics with and without attack.

FAQs

What question does the paper focus on regarding machine learning security defenses?

It asks for whom these defenses provide protection. Specifically, it evaluates whether security methods create performance inequities across different sub-populations.

What kinds of harm do the authors find in many proposed defenses?

The work reports that some methods can directly harm users through false rejection and unequal benefits from robustness training.

How does the speech recognition case study evaluate equality of defenses?

It measures how adversarial training and augmentation yield non-equal but complex protections across social subgroups defined by gender, accent, and age, and it compares randomized smoothing with neural rejection.

Aug 4, 2026

Upload by Himbo