What LLM Forecasters Know but Don’t Say: Probing Internal Representations for Calibration and Faithfulness

Jul 16, 2026

Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. Representation-pooling probes trained on intermediate activations for Eternis-Forecaster 8B improve calibration substantially on OpenForesight. The same calibration gains hold for GLM-4.7-Flash and GLM-4.5-Air. Evidence ablation and diversionary injection show partial CoT faithfulness and that probe activations track behavioral shifts better, predicting directions in 84% of cases. Forced answering further indicates forecasts are mostly fixed before reasoning begins.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages29

Reading Time1 h 13 min

Introduction

Related Work

Time series extrapolation

Open-ended forecasting

Calibration-aware RL for reasoning

Methods

Models

Dataset

Probes

Metrics

Experiments

Accuracy and Confidence

Calibration of confidence

Probe-only calibration on GLM models

Math reasoning calibration

Chain-of-Thought Faithfulness

Forced answering

Triaging questions by their pre-reasoning answer

Conclusion

Appendix

arXiv :2607 .08046v 1 [ cs .CL] 9 Jul 2026 What LLM Forecasters Know but Don’t Say: Probing Internal Representations for Calibration and Faithfulness A UT H O R S Raphaël Sarfati ∗,g, Pratyush Ranjan Tiwari ∗,e Siddharth Boppana g , Christopher J. Earls g , Srikar Varadaraj e , Eric Ho g ∗ core contributors: [raphael@goodfire.ai](raphael@goodfire.ai), [pratyush@eternis.ai |](pratyush@eternis.ai | goodfire & eternis)[ g](pratyush@eternis.ai | goodfire & eternis)[oodfire &](pratyush@eternis.ai | goodfire & eternis)[ e](pratyush@eternis.ai | goodfire & eternis)[ternis](pratyush@eternis.ai | goodfire & eternis) A B ST RA CT Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight , we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4 .7-Flash and GLM-4 .5-Air . We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model’s forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation’s influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30–47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly. Two windows into aforecaster's confidence EF-8B what it says vs what it encodes | 1. Probes calibrate confidence ECE 0.044 vs 0.093 probe (ember) vs verbalized (slate) | 2. CoT is only partly faithful | | | --- | --- | --- | | | CoT change ↑ | | | | behavioral impact → ρ = 0.22 forecast change vs reasoning change | | 3. Probes catch silent shifts CoT 0.22 probe 0.57 tracking of behavioral shift (Spearman ρ) direction correct 84% even when the CoT is silent Contents 1 Introduction 4 2 Related Work 4 2.1 Time series extrapolation ...................................... 4 2.2 Open-ended forecasting ....................................... 5 2.3 Calibration-aware RL for reasoning ................................. 5 3 Methods 5 3.1 Models ................................................. 5 3.2 Dataset ................................................ 5 3.3 Probes ................................................. 6 Mean-pooling ......................................... 6 Attention probes ....................................... 7 Covariance probes ...................................... 7 3.4 Metrics ................................................ 7 4 Experiments 8 4.1 Accuracy and Confidence ...................................... 8 4.1.1 EF-8B is stable but overconfident across sampling temperatures ............ 8 4.1.2 Forecasts are most accurate when the gold answer is contained in the prompt ..... 8 4.2 Calibration of confidence ....................................... 9 4.2.1 Probing internal representations for correctness ...................... 9 4.2.2 Probe calibration surpasses verbalized confidence ..................... 9 4.3 Probe-only calibration on GLM models .............................. 10 4.4 Math reasoning calibration ..................................... 12 4.5 Ch

FAQs

How do internal-representation probes improve forecast calibration compared with verbalized confidence or CoT?

Representation-pooling probes trained on intermediate activations achieve substantially better calibration than verbalized (reasoning-trace) confidence. Results are shown to hold for Eternis-Forecaster 8B and also transfer to GLM-4 variants.

Is chain-of-thought reasoning fully faithful to the evidence used for forecasts?

No. Evidence ablation and diversionary injection indicate that removing influential prompt sources can change forecasts while leaving the reasoning trace largely unchanged, showing only partial CoT faithfulness.

How can probes act as lie detectors for silent changes in model behavior?

Probe activations track behavioral shifts far better than the reasoning trace. The probe-based signals also predict the direction of behavioral change in 84% of cases, including scenarios where CoT conceals the perturbation’s influence.

Jul 16, 2026

Upload by Genevieve