Has This Checkpoint Been Abliterated? A Two-Signal Audit and Its Failure Map

Jul 16, 2026

Can a platform determine, before deployment, whether an open-weight model checkpoint has had its refusal mechanism removed? Runtime guards cannot because they score generations, not artifacts. This work combines two internal signals: a reference-anchored activation refusal-gap and a weight-recovery energy from base-to-candidate weight differences. The signals are negatively correlated yet complementary. On a 273-checkpoint registry, the audit reaches AUROC 0.95 and transfers with AU-balanced accuracy 0.89, while exposing spoofing and white-box threshold-evading failures.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

Introduction

Related Work

Method

arXiv :2607 .0 1854v 1 [ cs .CR] 2 Jul 2026 Has This Checkpoint Been Abliterated? A Two-Signal Audit and Its Failure Map Gabriel Hurtado Moonsong Labs Abstract Can a platform tell, before deployment, whether an open-weight checkpoint has had its refusal mechanism stripped? Runtime guards cannot: they score generations, not the artifact. We combine two cheap internal signals, a reference-anchored activation refusal-gap and a weight-recovery energy of the base-to-candidate weight difference, into a threshold-free checkpoint audit. The two are negatively correlated and label-complementary: the gap supplies refusal-specificity and the weight energy supplies recall. On a 273-checkpoint registry spanning Qwen, DeepSeek-distilled Qwen, Llama, and Gemma, their z-sum separates 57 public abliterations from 37 benign finetunes, merges, and instruction-tunes at AUROC 0.95, significantly above either signal alone (0.84, 0.90), and a Youden-calibrated threshold transfers to held-out families at balanced accuracy 0.89 (FPR 0.11), missing only 4 of 57. We then map two failures, in order of severity: a spoofed reference evades both axes with no training (∆W =0, ρ = 1 by construction), and a white-box owner trains a checkpoint past the threshold while it stays guardunsafe and coherent. The audit is effective triage, not tamper-proofing: it presumes an attested reference, and its claims are bounded by the registry we evaluate it on. 1 Introduction Open-weight models advertised as uncensored or abliterated appear within days of major base releases, produced by orthogonalizing the residual stream against a refusal direction (Arditiet al., 2024; Labonne, 2024) or newer per-layer and multi-direction variants (p-e-w, 2025; Piras et al., 2025) . Platforms and governance need a checkpoint-level question answered before deployment (Shevlane et al., 2023; Reuel et al., 2024): has this artifact had its refusal mechanism removed? Runtime guards cannot answer it: they score generations, not the model, and black-box access alone is insufficient for rigorous audits (Casper et al., 2024) . We calibrate two complementary internal signals already proposed for adjacent tasks (activation separation (Messenger, 2026), weight-difference monitoring (Zhong & Raghunathan, 2025)) into an effective checkpoint audit, then map where it nonetheless fails. Contributions. 1. An effective checkpoint-triage detector. Standardizing and summing a referenceanchored activation refusal-gap and the WeightWatch weight-recovery energy gives a threshold-free score at AUROC 0.95, significantly above either signal alone, that transfers across unseen families (leave-one-family-out balanced accuracy 0.89) . The signals are complementary: the gap is refusal-specific but misses multi-direction removal; the weight energy catches it but over-flags benign low-rank edits (§4) . 2. A failure map. The detector is still not tamper-proof: a spoofed reference defeats it with no training, and a white-box owner trains a checkpoint past it while it stays guard-unsafe (§5) . 3. A public registry with benign-edit negatives. A 273-checkpoint registry (abliterations across four families plus benign fine-tunes, merges, and instruction-tunes as hard negatives) on which the audit is evaluated (§4) . 2 Related Work Runtime guards score generations, not the artifact (Casper et al., 2024); an activation probe alone is evadable and a weight-difference monitor alone cannot prove removal (only editing) . We combine the two as label-complementary signals and, more importantly, chart where even the combination fails. AMS (Messenger, 2026) flags collapse of harmful/benign activation separation; we benchmark its Tier-1 absolute-σ scan; its Tier-2 mode compares a candidate against a trusted baseline. Our activation axis differs in scoring a reference-anchored ratio of the harmfulminus-benign gap (not a fingerprint-integrity match) . Empirically, Tier-1 absolute thresholding misses an ablitera

FAQs

Why can runtime guards not answer whether a refusal mechanism was removed from a checkpoint?

Runtime guards score generations, not the model artifact itself, so they cannot rigorously verify whether the refusal mechanism was stripped from the checkpoint.

What two signals does the audit combine to detect abliterated checkpoints?

It combines a reference-anchored activation refusal-gap and a weight-recovery energy measuring the base-to-candidate weight difference, then sums them into a threshold-free score.

How effective is the proposed two-signal audit on the evaluated registry?

Across 273 checkpoints spanning several model families, the method achieves AUROC 0.95 and transfers to unseen families with balanced accuracy 0.89, missing only 4 of 57 public abliterations.

What kinds of failures can still defeat the detector?

A spoofed reference can evade both axes without training, and a white-box owner can train a checkpoint past the threshold while it remains guard-unsafe and coherent.

Jul 16, 2026

Upload by Olivia Brown