The Objective Decides: When a Learned Dynamics Model Uses a Conserved Quantity

Jul 16, 2026

Upload by Aria

A linear probe that recovers a conserved quantity from a learned dynamics model’s activations is routinely treated as proof that the model uses that quantity. This work demonstrates the inference is unsound: across mechanical, circuit, and PDE systems, and on a 158M-parameter pretrained PDE foundation model, conserved invariants are linearly decodable with R2 near 1 while remaining causally inert for next-state prediction. The invariant becomes load-bearing only when the training objective rewards it, making deployment an objective property. A precise algebraic predicate governs when an invariant is deployed, and the deployment gap predicts OOD accuracy where decodability cannot.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Presence is not use

This is not a tautology

Measure deployment, not decodability

arXiv :2607 .03728v 1 [ cs .CE] 4 Jul 2026 THE OBJECTIVE DECIDES: WHEN A LEARNED DYNAMICS MODEL USES A CONSERVED QUANTITY Chih-Ting Liao & Xin Cao University of New South Wales [mill.liao@unsw.edu.au](mill.liao@unsw.edu.au) ABSTRACT A linear probe that recovers a conserved quantity from a learned dynamics model’s activations is routinely read as evidence that the model uses that quantity. We show this inference is unsound. Across mechanical, circuit, and partial-differentialequation (PDE) systems, and on a 158M-parameter pretrained PDE foundation model, energy and other invariants are linearly decodable at R2 ≈ 1 yet causally inert on next-state prediction: overwriting the decoded direction with a donor state’s value, single-step activation interchange, leaves the forward pass essentially unchanged (transfer-corr τ ≈0) . The same direction in the same representation becomes causally load-bearing (τ →+1) the moment the training objective rewards the invariant, so deployment is a property of the objective, not of the representation or the probe. We further show that when an invariant is deployed is governed by a precise algebraic predicate, its relation to the prediction output, by flipping a single invariant from inert to load-bearing by changing only the output’s algebra. Finally, the gap has teeth: across models that all decode the target at R2 =1 .00, the deployment gap forecasts out-of-distribution (OOD) accuracy (r=+0 .97) where decodability is blind. We argue that causal deployment, not decodability, is what interpretability should measure when the question is whether a model uses a piece of knowledge, and we give a cheap instrument for measuring it. 1 INTRODUCTION Does a neural network that predicts the physical world understand the quantities that govern it? The question is not academic. Learned dynamics models and PDE foundation models are increasingly used as fast surrogates for simulation in the physical sciences, and a growing literature asks whether they have internalized the conserved quantities, energy, momentum, charge, that a physicist would insist on. When a probe recovers such a quantity from a model’s activations, the field reads it asa yes: the model has learned, represents, and uses the invariant [1, 4, 24] . That inference is what licenses trusting the surrogate. This paper argues that the inference is wrong, and quantifies exactly how wrong. The field’s default answer comes from probing: fit a linear map from a model’s activations to a quantity of interest and read a high R2 as evidence that the model has learned and uses it. We show this reading conflates two claims that come apart, and give an instrument that separates them. Move 1: presence is not use. A probe that recovers ϕ at R2 ≈ 1 certifies that the information is present in the representation. It says nothing about whether the computation depends on it. These are different claims, and only the second predicts behavior. We make the distinction operational with a causal test, single-step activation interchange, and find that in a fully-observed learned dynamics model, energy is linearly decodable at R2 ≈1 and yet causally inert on next-state prediction: setting the decoded direction to any other in-distribution value changes the model’s one-step prediction by an amount statistically indistinguishable from a matched random direction. The model represents the invariant and does not use it. Move 2: this is not a tautology. A skeptic will object that the dissociation is trivial, a next-state predictor is trained on the microstate, so of course the invariant is redundant. Three facts make it non-trivial. (i) The same direction in the same representation flips from inert to causally load-bearing when only the training objective changes, with an identical probe and identical evaluation data; the effect is therefore a property of deployment, not of the representation or the probe. (ii) Whether an invariant is deployed is governed

FAQs

Why is high probe R2 not sufficient evidence that a model uses a conserved quantity?

High R2 indicates the information is present in the representation, but it does not show the computation depends on it. The paper finds invariants can be decodabIe yet causally inert for next-state prediction.

What causal test is used to separate decodability from use?

The paper applies a single-step activation interchange (donor-patch activation interchange), then measures change in the forward pass using a scale-free transfer-corr τ against a matched random-direction control.

What determines whether an invariant is actually deployed in the model?

Deployment is governed by a precise algebraic predicate relating the invariant to the prediction output. By changing only the output’s algebra, the invariant can flip from inert to load-bearing.

Jul 16, 2026

Upload by Aria