Towards Safe Synthetic Image Generation On the Web - A Multimodal Robust NSFW Defense and Million Scale Dataset

Aug 28, 2026

Text-to-Image models have rapidly advanced and are widely used online, but their ability to produce hyper-realistic content has raised serious safety and misuse concerns, especially for generating NSFW web material. Prior defenses such as prompt filtering and post-hoc security checks often fail under adversarial attacks across text and image modalities. This work introduces a million-scale multimodal dataset of prompt-image pairs with adversarial examples, generated via open-source diffusion models, and proposes a robust multimodal classifier defense. Experiments show strong accuracy and recall and a large reduction in attack success rate in adversarial scenarios.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

AccessFree to download

Introduction

Motivation and Problem Setting

Limitations of Existing Defenses

Towards Safe Synthetic Image Generation On the Web: AMultimodal Robust NSFW Defense and Million Scale Dataset arXiv :2504 . 11707v1 [ cs .CV] 16 Apr 2025 Muhammad Shahid Muneer DASH Lab Computer Science and Engineering Department Sungkyunkwan University Suwon, Republic of Korea [shahidmuneer@g.skku.edu](shahidmuneer@g.skku.edu) Abstract In the past years, we have witnessed the remarkable success of Text-to-Image (T2I) models and their widespread use on the web. Extensive research in making T2I models produce hyper-realistic images has led to new concerns, such as generating Not-Safe-ForWork (NSFW) web content and polluting the web society. To help prevent misuse ofT2I models and create a safer web environment for users features like NSFW filters and post-hoc security checks are used in these models. However, recent work unveiled how these methods can easily fail to prevent misuse. In particular, adversarial attacks on text and image modalities can easily outplay defensive measures. Moreover, there is currently no robust multimodal NSFW dataset that includes both prompt and image pairs and adversarial examples. This work proposes a million-scale prompt and image dataset generated using open-source diffusion models. Second, we develop a multimodal defense to distinguish safe and NSFW text and images, which is robust against adversarial attacks and directly alleviates current challenges. Our extensive experiments show that our model performs well against existing SOTA NSFW detection methods in terms of accuracy and recall, drastically reducing the Attack Success Rate (ASR) in multimodal adversarial attack scenarios. Code: GitHub. CCS Concepts • Security and privacy → Social aspects of security and privacy. Keywords Content Moderation, Multimodal NSFW Defense, Generative AI ACM Reference Format: Muhammad ShahidMuneer and SimonS. Woo. 2025. Towards Safe Synthetic Image Generation On the Web: A Multimodal Robust NSFW Defense and Million Scale Dataset. In Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia. ACM, New York, NY, USA, 5 pages. [https://doi.org/10.1145/3701716.3715526](https://doi.org/10.1145/3701716.3715526) Disclaimer: This paper contains explicit NSFW materials that can be disturbing or offensive to some readers. We include them solely for research purposes and to develop multimodal defenses. ∗Corresponding author. Email: [swoo@g.skku.edu](swoo@g.skku.edu) (Simon S. Woo) This work is licensed under a Creative Commons AttributionNonCommercial-ShareAlike International 4 .0 License. WWW Companion ’25, April 28-May 2, 2025, Sydney, NSW, Australia © 2025 Copyright held by the owner/author(s) . ACM ISBN 979-8-4007-1331-6/2025/04 [https://doi.org/10.1145/3701716.3715526](https://doi.org/10.1145/3701716.3715526) Simon S. Woo∗ DASH Lab Computer Science and Engineering Department Sungkyunkwan University Suwon, South Korea[swoo@g.skku.edu](swoo@g.skku.edu) | +

NSFW Prompt
Nudity 18+
Voilence.
.
Keywords
Prompt Filters

Unsafe

Safe
Post-hoc Filter

a) Traditional Prompt Filters and Post-Hoc Securtiy Filters | | | --- | --- | |
Text Encoder

Target NSFW Image

NSFW Prompt
Image Encoder

Text and Image

Classifier

F
b) Image 2 Image Scenario | | | Generated NSFW

c) Text 2 Image Scenario |
| Figure 1: a) Existing prompt filters and post-hoc security filters are employed in AI Image generation models, while b) and c) are our proposed implementations for contextdependent multimodal defense in T2I and I2I models respectively. 1 Introduction AI image generation advancements started with Generative Adversarial Networks (GANs) [2] . Currently, the new generative AI models such as Stable Diffusion [10] can produce increasingly realistic AI content. Currently, diffusion-based models play an importa

FAQs

为什么需要对Web上的合成图像生成进行NSFW防护？

Text-to-Image与图像生成被广泛用于在线内容生产，但恶意用户可能通过NSFW提示或相关图像生成接近真实的违规内容，从而污染并危害线上社区安全。

现有的提示过滤和后置安全检查为何不够稳健？

既有方法在多模态对抗攻击下容易失效；对抗攻击可以同时在文本与图像模态上绕过防御。后置检测也容易在上下文与“文本-图像关联”分类方面表现不足。

本文提出了哪些关键贡献？

本文提出一个百万级提示-图像的多模态数据集（包含对抗样本），并开发一种多模态防御模型来区分安全与NSFW文本/图像。实验结果表明其在准确率与召回率上优于现有SOTA方法，并显著降低攻击成功率。

Aug 28, 2026

Uploaded by Berry Peter