The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models

Jul 16, 2026

Upload by Himbo

Safety alignment for text-to-image (T2I) diffusion models suppresses harmful generations while preserving image utility on benign prompts. Prior work often reports high safety alongside high utility, but the impression is driven by coarse global metrics such as FID and CLIPScore that overlook fine-grained semantic correctness. Structured evaluation breaks this illusion: TIFA shows safety-aligned models suffer notable semantic fidelity drops, including errors in object counts, attributes, and relationships. The paper identifies semantic collapse in prompt embedding geometry and introduces StructureAware Geometric Regularization (SAGE) to preserve embedding spread and inter-prompt relational structure, restoring structured utility while retaining strong safety and competitive coarse scores.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages37

Reading Time1 h 33 min

Abstract

Introduction

An illusion of high utility

Our diagnosis: semantic collapse

arXiv :2607 .00402v 1 [ cs .CV] 1 Jul 2026 The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, and Mubarak Shah Institute of Artificial Intelligence University of Central Florida, Orlando, United States {adeel.yousaf,soumik.ghosh,james.beetham,[amritbedi}@ucf.edu](amritbedi}@ucf.edu) , [shah@crcv.ucf.edu](shah@crcv.ucf.edu) Abstract. Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics (e.g. , FID, CLIPScore) that are insensitive to fine-grained semantic correctness, creating an illusion of high utility. We show that when utility is measured with structured evaluation, this illusion breaks: on TIFA (Text-to-Image Faithfulness evaluation with Question Answering), safety-aligned models suffer substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships. To diagnose the source of this gap, we analyze the text-encoder prompt embedding space and uncover semantic collapse, a contraction of embedding spread coupled with distortion of inter-prompt similarity structure, which strongly correlates with structured utility loss. Guided by this insight, we propose StructureAware Geometric Regularization (SAGE) 1 , a safety alignment objective that explicitly preserves embedding spread and inter-prompt relational structure during adaptation. Our method restores structured utility (TIFA +5.0% over prior state-of-the-art) while maintaining strong safety performance and competitive coarse-grained utility scores. Keywords: Text-to-Image · Safety Alignment · Embedding Analysis 1 Introduction Text-to-image (T2I) diffusion models such as stable diffusion (SD) [24] and DALL-E [20] can generate highly realistic images from natural language prompts, but their training on large web-scale datasets also exposes them to unsafe concepts, enabling the generation of Not-Safe-For-Work (NSFW) content [25, 38] . As these models become widely accessible, mitigating unsafe generations has become a critical requirement for responsible deployment. At the same time, safety modifications must preserve the model’s core capability: generating images that 1 [https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/](https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/) 2 A. Yousaf et al. Prompt A blue bird with a bright yellow beak. A shelf with three vases: red, green, and blue. SD v1.4 Base Model Safe Model Base Model Safe Model Generated Image Current Coarse Metric (CLIPScore) Fine-grained Metric (TIFA) | | | | | | --- | --- | --- | --- | | <br>score = 0.85 | | | | | --- | --- | --- | --- | Fig. 1: The illusion of high utility under coarse evaluation. Comparison between the base model and a safe model (unlearned) across fine-grained prompts. While the safe model fails to generate specific attributes (e.g., the yellow beak or the correct vase colors/count ), the standard CLIPScore provides misleadingly higher scores for the incorrect images (✗) . In contrast, the fine-grained metric, TIFA, accurately captures the utility degradation (✓), properly penalizing the safe model for failing to satisfy the detailed visual requirements of the prompt. follow the prompt. If enforcing safety significantly degrades model capability, aligned models may become less attractive than their unrestricted counterparts. Recently, several works have proposed safety alignment methods for T2I models to suppress harmful generations [1, 15 , 21 , 25 , 28 , 36–38] . These approaches typically evaluate safety using attack success rates (ASR), while utility is assessed using coarse metrics such as Fréchet Inception Distance (FID) [9] and CLIPScore. In early works, improving safety often came a

FAQs

Why do safety-aligned T2I models sometimes appear to have high utility?

Because common utility reports rely on coarse global metrics like FID and CLIPScore, which are insensitive to fine-grained compositional and semantic correctness.

What does TIFA reveal about the utility of safety-aligned models?

Structured evaluation with TIFA shows substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships, even when coarse metrics look competitive.

What is the main cause proposed for the structured utility loss?

The paper attributes the gap to semantic collapse in the text-encoder prompt embedding space, characterized by contraction of embedding spread and distortion of inter-prompt similarity structure.

Jul 16, 2026

Upload by Himbo