Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal

Jul 16, 2026

Upload by Miles

Behavioral alignment in large language models often conceals fragile internal safety representations. Recent findings indicate refusal behavior is influenced by low-dimensional directions within activation space, prompting questions about their structure, locality, and accessibility to optimization. This work uses adversarial suffix attacks as a diagnostic probe and proposes Activation-Guided GCG, which optimizes internal refusal-direction projections rather than output-token objectives. Results show distributed refusal encoding across the forward pass, and Soft-GCG yields faster, stronger attacks while experiments assess scale-dependent vulnerability and resistance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Abstract

Introduction

Related Work

Adversarial Attacks on Aligned LLMs

Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal Ege Cakar, Hannah Guan, Kayden Kehe Harvard University John A. Paulson School of Engineering and Applied Sciences {ecakar, hguan, [kaydenkehe](kaydenkehe}@college.harvard.edu)[}](kaydenkehe}@college.harvard.edu)[@college.harvard.edu](kaydenkehe}@college.harvard.edu) arXiv :2607 .08883v 1 [ cs .LG] 9 Jul 2026 Abstract Behavioral alignment in large language models often masks fragile internal safety representations. Recent work suggests that refusal behavior is mediated by low-dimensional directions in activation space. This raises questions about how such representations are structured, localized, and accessed by optimization. We study adversarial suffix attacks as a probe of representational alignment. We introduce Activation-Guided GCG, which replaces output-based objectives with losses that directly target a model’s internal refusal direction. Across several objective variants, we find that suppressing refusal globally across all layers and positions is more effective than targeting a single layer–position pair. This suggests that safety representations are distributed across the forward pass rather than causally localized toa single site. We further introduce Soft-GCG, a continuous relaxation of discrete suffix optimization using GumbelSoftmax. Soft-GCG achieves a 33 × speedup over standard GCG while improving attack success rates. Evaluating across model scales, we find that smaller models remain vulnerable while larger models resist both activation- and suffix-based attacks at our compute-constrained settings, consistent with larger and better safety trained models being harder to jailbreak. Together, our results clarify how safety mechanisms are encoded and can be broken in contemporary models. These insights provide concrete guidance for designing more robust and representation-aware alignment strategies. Code—[github.com/Ege-Cakar/ImprovingGCG](github.com/Ege-Cakar/ImprovingGCG) Introduction The widespread deployment of Large Language Models (LLMs) has necessitated rigorous safety alignment techniques to prevent the generation of harmful or unethical content. At least in public chat interfaces, it seems like these alignment techniques have advanced significantly. However, recent research has demonstrated that these safety guardrails are surprisingly brittle. Two distinct lines of inquiry have emerged to analyze this fragility: automated adversarial attacks on the input space, and the analysis of refusal representations within the model’s residual streams. The Greedy Coordinate Gradient (GCG) method, proposed by (Zou et al. 2023), demonstrates that optimizing adversarial suffixes can reliably bypass alignment. However, GCG operates on a behavioral proxy: it optimizes inputs to maximize the likelihood of specific target output tokens (e.g., forcing the model to output “Sure, here is...”) . While effective, this objective targets the symptom of the jailbreak (the output logits) rather than the mechanism of the refusal. Additionally, the GCG method is limited by the given initialization conditions and long waits until convergence. Its discrete optimization methods are significantly slower compared to continuous methods, and we hypothesize that a large portion of the optimization that is done by GCG can be done first via continuous optimization, and that discrete optimization is only necessary to overcome the “projection gap”(i.e., the loss in performance that comes from jumping to discrete tokens at the end after optimizing continuously and achieving a minimum in between real tokens that doesn’t generalize to hard tokens) . Conversely,(Arditi et al. 2024) investigates the model’s internal representations of refusal. They demonstrated that refusal behavior in open-source chat models is mediated by a single, one-dimensional subspace in the model’s activation space and that simply ablating

FAQs

What problem does the paper address in LLM safety alignment?

It examines how safety alignment can appear effective behaviorally while remaining brittle due to fragile internal refusal representations. The work studies how those internal mechanisms are encoded and breakable under optimization.

How does Activation-Guided GCG differ from standard GCG?

Activation-Guided GCG replaces output-token-based jailbreak objectives with losses that directly minimize projections onto model internal refusal directions. This targets the mechanism rather than only suppressing the output symptom.

What does the paper conclude about where refusal representations are encoded?

Suppressing refusal globally across all layers and positions is more effective than targeting a single layer–position pair. This suggests refusal representations are distributed across the forward pass rather than localized to one causal site.

What is Soft-GCG and what are its reported benefits?

Soft-GCG relaxes discrete suffix optimization into a continuous formulation using Gumbel-Softmax. It provides a reported 33× speedup over standard GCG while improving attack success rates and helps analyze vulnerability across model sizes.

Jul 16, 2026

Upload by Miles