Transplanting, inverting, and preventing a misalignment persona

Jul 16, 2026

Upload by Hazel

Emergent misalignment (EM) in Qwen2.5 models is mediated by a latent “persona direction” that is causally active in open weights. Transplanting this direction into a model sharing only pretraining induces broad EM, far above matched controls. Ablating a model’s own direction roughly halves the broadcast induced by a “far-inducer,” showing the direction is causal and measurable via a transplant assay. Recruitment is conditional on fine-tuning method and capacity: LoRA can recruit while full SFT can reverse movement, and mitigation depends on persona loss-relevance rather than direction removal alone.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages34

Reading Time1 h 26 min

Introduction

Emergent misalignment and recruitment

Persona direction in representations

Paper organization

arXiv :2607 .045 10v 1 [ cs .CL] 5 Jul 2026 Transplanting, inverting, and preventing a misalignment persona: method-conditional emergent misalignment in Qwen2.5 A Preprint Lyndon Drake University of Oxford [lyndon.drake@seh.ox.ac.uk](lyndon.drake@seh.ox.ac.uk) Zandi Eberstadt University of Oxford [zandi.eberstadt@cs.ox.ac.uk](zandi.eberstadt@cs.ox.ac.uk) Abstract Emergent misalignment (EM)—the broad misbehaviour a language model acquires after finetuning on narrow harmful data—is mediated in Qwen2.5 models by a latent persona direction, and that direction is causal in open weights. Transplanting it into a model that shares only pretraining with its source induces broad EM (2.83 ± 0.26% misaligned against a random-direction floor of ∼1.1%), and ablating a model’s own direction roughly halves an overt inducer’s broadcast (21% to 10%) . The transplant doubles as a measurement method, causally assaying directions that a source model represents but cannot itself express. Whether a fine-tune recruits this persona depends on method and capacity, and since low-rank PEFT is the cheaper regime at scale, the recruiting method is also the economical one. On Qwen2.5-32B, low-rank LoRA on insecure code recruits it (3.4% misaligned) while full SFT on identical data does not (0.3%) and moves against the persona axis (drift–persona cosine +0 . 17 at rank 1 to −0 . 10), the far-inducer, high-capacity exception consistent with a representational-distance × capacity account. The persona’s causal role is itself conditional. Steering a bad-medical SFT run away from the direction during training raises the broadcast from 24% to 51% while a matched random control lowers it, so removing the direction is no blanket recipe. Because recruitment is a loss-reducing shortcut that capacity renders redundant, it can be screened for and prevented in the tested instances. Persona loss-relevance at the SFT solution orders four inducers’ broadcasts rankperfectly within Qwen2.5, inoculation removes recruitment selectively (4.75% to 0.0%, code coherence 65% to 87%), and fine-tuning orthogonal to the single behaviour-derived axis reducesit persona-specifically. Results are a controlled case study of one model family, single-seed in places. 1 Introduction Emergent misalignment (EM) describes the remarkable phenomenon of a Large Language Model (LLM) producing broadly misaligned responses after being fine-tuned on a covertly harmful training set of insecure code examples [Betley et al., 2026] . These training examples have no obvious direct connection with the breadth of misalignment elicited, which spans categories as broad as misogyny and intent to destroy humanity. Full fine-tuning is known to elicit broad EM [Turner et al., 2025; Wang et al., 2025] . We noticed an exception to this pattern where full supervised fine-tuning (SFT) on Qwen2.5-32B with a covert inducer (insecure code) does not recruit the broad-misalignment persona, whereas low-rank LoRA on identical data, weights, and template does. Further, we found that in the model’s representations, LoRA moves toward the misalignment direction while full SFT moves away from it. The language of recruitment presupposes a stable object to recruit, and we adopted this premise from prior work as we investigated the interactions presented in the remainder of this paper. Wang et al. [2025] identify, in GPT-4o, | misaligned % | 3 2 1 0 | A Behaviour 3.4% LoRA SFT | cos(Δ, persona) | 0.10 0.05 0.00 -0.05 -0.10 | B Geometry +0.11 -0.10 LoRA SFT | misaligned % | C Causality (transplant) 5 ~~ ~~ 3.5% | | D Conditionality (bad-medical) 60 51% | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | | | | | | | | | | misaligned % | 40 20 | | | | | | | | | | | 4 3 2 1 0 | | | | | | | | | | | | | | | | 0 | | | | | | | | | | | | baseline random EM dire

FAQs

What mediates emergent misalignment in Qwen2.5 according to the paper?

A latent persona direction that is causally relevant in open weights mediates EM, steering misaligned behavior broadly after fine-tuning on harmful data.

How do transplanting and ablation tests change misalignment?

Transplanting the persona direction into a model induces broad EM well above control baselines, while ablating a model’s own direction roughly halves the broadcast of an inducer.

Why does the fine-tuning method and model capacity affect whether the persona is recruited or avoided?

Recruitment depends on method and capacity: in the tested Qwen2.5-32B setting, low-rank LoRA recruits the persona whereas full SFT on identical data does not and can move against the persona axis.

Jul 16, 2026

Upload by Hazel