Mind the Gap - Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis

Aug 20, 2026

Upload by Violet

Adversarial attacks pose a serious risk to the integrity of machine learning models, and query-based black-box attacks are especially practical because they can craft malicious perturbations without knowing the victim model’s internals. This paper introduces a framework to detect whether adversarial noise generation is underway. Rather than monitoring input-space states, it learns adversarial patterns in an update-similarity space using a new Delta Similarity (DS) metric. Evaluations against eight state-of-the-art attacks, including adaptive evasion attempts, show improved robustness with higher specificity and sensitivity and lower false positives.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

AccessFree to download

Introduction

Threat models: white-box vs black-box

Query-based black-box attacks and existing defenses

Proposed perspective: detect update patterns

Delta Similarity and evaluation setup

This CVPR paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis Jeonghwan Park, Niall McLaughlin, Ihsen Alouani Queen’s University Belfast, United Kingdom [jeonghwan.park@uk.thalesgroup.com](jeonghwan.park@uk.thalesgroup.com), {n.mclaughlin, [i.alouani](i.alouani}@qub.ac.uk)[}](i.alouani}@qub.ac.uk)[@qub.ac.uk](i.alouani}@qub.ac.uk) Abstract Adversarial attacks remain a signi􀀂cant threat that can jeopardize the integrity of Machine Learning (ML) models. In particular, query-based black-box attacks can generate malicious noise without having access to the victim model’s architecture, making them practical in real-world contexts. The community has proposed several defenses against adversarial attacks, only to be broken by more advanced and adaptive attack strategies. In this paper, we propose aframework that detects if an adversarial noise instance is being generated. Unlike existing stateful defenses that detect adversarial noise generation by monitoring the input space, our approach learns adversarial patterns in the input update similarity space. Infact, we propose to observe a new metric called Delta Similarity (DS), which we show it captures more ef􀀂ciently the adversarial behavior. We evaluate our approach against 8 state-of-the-art attacks, including adaptive attacks, where the adversary is aware of the defense and tries to evade detection. We 􀀂nd that our approach is signi􀀂cantly more robust than existing defenses both in terms of speci􀀂city and sensitivity.1 1. Introduction Adversarial attacks have been investigated as a critical threat to the trustworthiness of ML systems. Two main threat models have been considered in the literature: whitebox and black-box. The white-box setting assumes the adversary has access to the model parameters, architecture and gradient. This setting has been investigated thoroughly to provide a worst-case vulnerability analysis [14, 15, 24] . The second scenario, i.e., black-box, is more practical in real-world contexts and corresponds to an adversary with limited access to the victim model, restricted to inputs and outputs. This threat model is particularly relevant in Machine Learning-as-a-Service (MLaaS) settings, where models can be queried remotely by clients through APIs. A 1 Code is available at [https://github.com/jpark04-qub/GWAD](https://github.com/jpark04-qub/GWAD) Figure 1 . A high-level illustration of our intuition. The sequence of malicious queries to generate an adversarial example has a different pattern than benign queries; Attack steps require random vector updates for gradient estimation. malicious actor generates adversarial perturbations by iteratively querying the model and analyzing the corresponding outputs. Despite the restricted access scenario, many querybased black-box attacks have been proposed [6, 8, 9, 19], showing a high ef􀀂ciency in fooling ML models. Most existing defenses against adversarial attacks frame the problem in a ”post-mortem” manner, i.e., assuming adversarial examples have already been generated [5, 16, 24, 27] . However, a recent line of work speci􀀂cally focused on query-based black-box settings suggests to detect adversarial attacks in the making [7, 10, 23] . These approaches leverage the adversary’s need to query the victim model, proposing to monitor input queries for anomalies in the input space that could indicate attempts to generate adversarial examples. For instance, Stateful Detection [7] identi􀀂es adversarial attack attempts by tracking the similarity (statefulness) between input queries, 􀀃agging anomalous queries that deviate signi􀀂cantly from typical behavior patterns. Similar methods, such as Blacklight [23] and PIHA [10], also rely on statefu

FAQs

What problem does the paper address?

It addresses detecting query-based black-box adversarial attacks while the adversarial example is being generated, rather than only after the adversarial example exists.

How does the proposed method differ from existing defenses?

Existing stateful defenses mainly monitor anomalies in the input space, whereas this work detects adversarial behavior by learning patterns in an input update similarity space using Delta Similarity (DS).

What metric is introduced and what is its purpose?

The paper introduces Delta Similarity (DS) to capture how updates in a sequence of queries relate over time, providing a more efficient indicator of adversarial behavior.

Aug 20, 2026

Upload by Violet