One mechanism for many mental spaces: a shared router over a value slot in language models

Jul 16, 2026

Upload by Ezra

Language models can express discourse contexts beyond the actual world—such as portraits, beliefs, memories, and hypotheticals—where one entity takes different values. Formal semantics separates these contexts by differing logics, while mental-space theory treats them as a single space-building operation. This work asks which organization a transformer implements, and finds a mechanistic unification: a shared router/slot format, where a low-rank space index selects which attributed content is read. The mechanism transfers across space types, supports inference, and composes new routers when new spaces are built, without special separation for belief.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

Introduction

Mental spaces and why contexts differ

Organization of the local context

Mechanistic representation in language models

One mechanism for many mental spaces: a shared router over a value slot in language models Oliver Steele [steele@osteele.com](steele@osteele.com)[ ](steele@osteele.com)Tsinghua University NYU Shanghai Jiangtao Wen [jw9263@nyu.edu](jw9263@nyu.edu)[ ](jw9263@nyu.edu)NYU Shanghai Yuxing Han [yuxinghan@sz.tsinghua.edu.cn](yuxinghan@sz.tsinghua.edu.cn)[ ](yuxinghan@sz.tsinghua.edu.cn)Tsinghua University 2026-06-29 Abstract Language builds discourse contexts other than the actual: a painting, a belief, a memory, a hypothetical. Each is a mental space in which the same entity can take a different value, as when a flower is red in reality but purple in a portrait. Formal semantics keeps these contexts apart because their logics differ (modal, temporal, doxastic, depictive); Fauconnier’s mental-space theory treats them as one space-building operation. We ask which of these a transformer language model implements, and find a mechanistic version of Fauconnier’s unification. The model uses one router/slot format across the inventory: a reusable value slot stores attributed content, and a causally manipulable router (the space index) selects which space is read. A subspace trained with Distributed Alignment Search to control one space type, counterfactual, belief, fictional, or temporal, also controls the others, well above a random floor, on three model families; belief, which formal semantics marks as a distinct case, is not specially separated. The router is low-rank, composes additively with entity identity, and acts through a few late-layer heads. Two further results show the mechanism drives inference and composes: a subspace trained on a rule-derived conclusion flips what the model infers while dissociating from what it reports, and composing space-builders mints a fresh router over the shared slot. This paper establishes the cross-type generality. A companion paper develops belief in depth, because of its special status in philosophy, psychology, and linguistics (epistemology, theory of mind, and propositional attitude reports) . Keywords mental spaces · binding · counterfactual · theory of mind · mechanistic interpretability · distributed alignment search 1 Introduction A capable language model keeps a flower’s real color apart from its color in a portrait, its present apart from its past, and what is true apart from what a character believes. We show it does so with a single, causally manipulable space index: a low-rank subspace that tags which discourse space a value belongs to. Formal semantics keeps these contexts apart because their logics differ; Fauconnier’s mental-space theory unifies them as one space-building operation. The model implements a mechanistic version of that unification: a subspace trained on one space type controls the others (transfer index 0.71–0.89 across model families, § 4.8), while the semantic distinctions survive as a coarse taxonomy inside the shared subspace (§ 4.11) . This is evidence for a shared representational format in LMs, not an adjudication of the semantics of English. Figure 1: One entity (the flower) takes a different value in each discourse space, and a single shared low-rank subspace, the space index (router), tags which space each value belongs to. A query names a space and the router selects its value for the output. We show this one router is causally shared across space types (§ 4.8), used in reasoning (§ 4.7), and minted afresh when a sentence builds a new space (§ 4.10) . Put plainly: whenever a sentence opens a non-actual context, a portrait, a belief, a memory, a hypothetical, the model files each value with a tag for which context it belongs to, and one kind of tag serves them all. The tag is separate from the value it marks, and the same tagging machinery is reused whether the context is a painting, a belief, or yesterday. The rest of the paper localizes that tag, shows it is one shared subspace across context types, and shows the model mints

FAQs

What discourse contexts does the paper focus on?

The paper focuses on non-actual contexts like portraits, beliefs, memories, and hypotheticals, where a shared entity can take different values depending on the context.

How does the proposed mechanism unify multiple mental-space types?

It uses a shared router/slot format: one reusable value slot stores attributed content, while a causally manipulable space index selects which space’s value is read.

Does the mechanism treat belief as a special case?

No. Although formal semantics marks belief as distinct, the model’s performance shows belief is not specially separated within the shared representational format.

Jul 16, 2026

Upload by Ezra