ImageNet-Patch - A dataset for benchmarking machine learning robustness against adversarial patches

Aug 4, 2026

Adversarial patches are contiguous pixel blocks that drive machine-learning models to misclassify input images, but optimizing them is computationally expensive and depends on careful hyperparameter tuning, which can weaken robustness evaluations. ImageNet-Patch provides a benchmark dataset of pre-optimized patches created using a transferable, state-of-the-art attack, then randomly rotated and translated before being applied to ImageNet data. The benchmark evaluates 127 models and further validates patch effectiveness in physical settings by printing and deploying them on real objects.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

Problem: adversarial patches and robustness evaluation

Proposed solution: ImageNet-Patch dataset

Benchmark construction and physical-domain validation

Pattern Recognition 134 (2023) 109064 Contents lists available at ScienceDirect Pattern Recognition journal [homepage: www.elsevier.com/locate/patcog](homepage: www.elsevier.com/locate/patcog) | ImageNet-Patch: A dataset for benchmarking machine learning robustness against adversarial patches | | | | | --- | --- | --- | --- | | Maura Pintora,c, Daniele Angionia, Angelo Sotgiua,c, Luca Demetrio b,c, Ambra Demontisa,∗, Battista Biggioa,c, Fabio Rolib,c
a University of Cagliari, Italy b University of Genova, Italy c Pluribus One, Italy | | | | | a r t i c l e i n f o | | a b s t r a c t | | | Article history:
Received 1 March 2022
Revised 7 July 2022
Accepted 20 September 2022
Available online 23 September 2022 | | Adversarial patches are optimized contiguous pixel blocks in an input image that cause a machinelearning model to misclassify it. However, their optimization is computationally demanding, and requires careful hyperparameter tuning, potentially leading to suboptimal robustness evaluations. To overcome these issues, we propose ImageNet-Patch, a dataset to benchmark machine-learning models against adversarial patches. The dataset is built by ﬁrst optimizing a set of adversarial patches against an ensemble of models, using a state-of-the-art attack that creates transferable patches. The corresponding patches are then randomly rotated and translated, and ﬁnally applied to the ImageNet data. We use ImageNet-Patch to benchmark the robustness of 127 models against patch attacks, and also validate the effectiveness of the given patches in the physical domain (i.e., by printing and applying them to real-world objects). We conclude by discussing how our dataset could be used as a benchmark for robustness, and how our methodology can be generalized to other domains. We open source our dataset and evaluation code at [https://github.com/pralab/ImageNet-Patch](https://github.com/pralab/ImageNet-Patch).
© 2022 Elsevier Ltd. All rights reserved. | | | Keywords:
Adversarial machine learning
Adversarial patches
Neural networks
Defense
Detection | | | | 1. Introduction Understanding the security of machine-learning models is of paramount importance nowadays, as these algorithms are used ina large variety of settings, including security-related and missioncritical applications, to extract actionable knowledge from vast amounts of data. Nevertheless, such data-driven algorithms are not robust against adversarial perturbations of the input data [1– 4] In particular, attackers can hinder the performance of classiﬁcation algorithms by means of adversarial patches [5], i.e., contiguous chunks of pixels which can be applied to any input image to cause the target model to output an attacker-chosen class. When embedded into input images, adversarial patches produce out-ofdistribution samples. The reason is that the injected patch inducesa spurious correlation with the target label, which is likely to shift the input sample off the manifold of natural images. Adversarial patches can be printed as stickers and physically placed on real objects, like stop signs that are then recognized as speed limits [6], and accessories that camouﬂage the identity of a person, hiding ∗ Corresponding author. E-mail address: [ambra.demontis@unica.it](ambra.demontis@unica.it) (A. Demontis). their real identity [7,8]. Therefore, the evaluation of the robustness against these attacks is of the uttermost importance, as they can critically impact real-world applications with physical consequences. The only way to assess the robustness of a machine-learning system against adversarial patches is to generate and test them against the target model of choice. Adversarial patches are created by solving an optimization problem via gradient descent. However, this process is costly as it requires both querying the target model many times and computing the back-propagation algorithm until convergence is reached. Hence

FAQs

What problem does ImageNet-Patch address in robustness evaluation?

It addresses the high computational cost and hyperparameter sensitivity of optimizing adversarial patches, which makes robustness evaluation slow and potentially suboptimal. The dataset replaces costly per-evaluation optimization with pre-optimized patches.

How are the ImageNet-Patch adversarial patches constructed?

The patches are optimized against an ensemble of models using a transferable adversarial patch attack, so they work across different model architectures. Each patch is then randomly rotated and translated before being applied to ImageNet images.

How is the benchmark validated beyond digital image testing?

The method validates effectiveness in the physical domain by printing the patches and applying them to real-world objects. This checks whether the intended misclassification behavior survives uncontrolled acquisition and environmental conditions.

Aug 4, 2026

Upload by Olivia Brown