Open-Weight Masked Introspection - Measuring What Language Models Can Report About Their Own Computation

Sep 15, 2026

The paper investigates whether frontier language models can introspect on internal states and report reliably about changes to their own computation. Tests on eight open-weight models across seven families show reports do not exceed chance at detecting whether computation was altered. To measure this capability, the authors introduce Open-Weight Masked Introspection (OWMI), which intervenes on internal computational objects and evaluates report informativeness against sham, impact-matched perturbations, and a text-only observer. Large-scale results indicate persistent failure, while fine-tuning and probing demonstrate the needed information can be extracted from activations, suggesting oversight methods must validate against internal references.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages45

Reading Time1 h 53 min

AccessFree to download

Abstract

Introduction

Intervention reportability concept

OWMI measurement framework

Conclusion and release

arXiv : 2608 .20569v 1 [cs .AI] 20 Aug 2026 Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation Emilio Ferrara University of Southern California [emiliofe@usc. edu](emiliofe@usc. edu) August 24, 2026 Abstract Are frontier models able to introspect about their internal states? Recent work suggests that under certain conditions a complex enough model can audit its own internals, call out what changed, and report back confidently about it. We tested that claim on eight open-weight models from seven families and found no such ability: asked whether their own computation had been altered, none answered better than chance. To test it we built Open-Weight Masked Introspection (OWMI), a framework that intervenes on internal computational objects such as residual-stream sites, attention heads and sparse-autoencoder features, then interrogates the model about the change against the null conditions an answer has to beat: sham runs where nothing was altered, impact-matched random perturbations, and a text-only observer that sees only the visible output. OWMI attaches to benchmarks already in use. Over 78,000 measurements, no model’s report discriminates a real intervention from a sham beyond chance (AUROC ≈ 0.5007), and an equivalence test bounds the effect below 0.15 percentage points of AUROC. Surprisingly, all the information needed to make that assessment is in the models. A model fine-tuned to report this class of intervention reaches near-perfect recovery on held-out directions (AUROC ≈ 1.0), and a linear probe recovers intervention presence from the same activations at between 75% and 95.8% accuracy, sharpening to no held-out error at the last layer before the model speaks. In one model the signal surfaces in the confidence rather than the words: its yes-or-no report never varies, while the confidence attached to that report separates intervention from sham at AUROC 0.647. The failure sits in the path from internal state to verbal report, so oversight practices that read a model’s own testimony, from chain-of-thought monitoring to self-critique, need validating against an internal reference rather than against that testimony. While our results show the inability of current open-weight models to introspect, the debate is not settled for future models. We release OWMI as a library so that this emerging ability can be measured as it develops: [https://huggingface.co/emilioferrara/owmi](https://huggingface.co/emilioferrara/owmi). 1 Introduction Can a language model report what happened inside the computation that produced its answer? A growing set of oversight practices assumes it can. Chain-of-thought monitoring assumes that stated reasoning tracks performed reasoning. Self-critique assumes that a model can audit the process behind its output. Confidence elicitation assumes that a model can read its own uncertainty [1–3] . Recent intervention studies encourage the assumption: inject a concept into a model’s activationsand it will sometimes notice that something is wrong, and occasionally name it [4] . We tested that capability systematically on eight open-weight models from seven laboratory families and reached a confident conclusion: no model reports our controlled interventions at a magnitude that supports detection. Nothing in the benchmark ecosystem would have caught this, and the reason is structural. Benchmarks grade the answer a model gives. None of them grades what the model can say about the computation that produced the answer, because none of them knows what that computation was. Without a reference, a report that is true and a report that merely sounds true score identically, anda model that always claims to notice something does as well as a model that genuinely does. The intervention studies supply that reference and establish the sham condition that goes with it; what they leave open is how much of a report survives once generic disruption a

FAQs

What is the core question the paper asks about language models?

Whether a language model can report what happened inside the computation that produced its answer, rather than only reflecting the stated reasoning or visible output.

What did the authors find when testing eight open-weight models?

When asked if their own computation had been altered, none answered better than chance in detecting the interventions.

How does OWMI measure “intervention reportability”?

OWMI intervenes on specific internal computational objects, then compares the model’s report against multiple null conditions: sham runs, impact-matched random perturbations, and a text-only observer bound.

Sep 15, 2026

Uploaded by Clementine