Knowing Before Saying - LLM Representations Encode Information About Chain-of-Thought Success Before Completion

Sep 4, 2026

Investigates whether zero-shot chain-of-thought (CoT) success can be predicted before the model finishes generating. A probing classifier built on LLM internal representations predicts correctness even before a single token is produced, indicating that crucial reasoning information is present in early representations. In contrast, a BERT-style baseline using only generated tokens performs worse, implying reliance on shallow linguistic cues. Later reasoning steps do not always improve prediction, and similarity across steps suggests early termination may preserve benefits. Early-stopping experiments confirm truncated CoT can outperform no-CoT while leaving a gap to full reasoning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Abstract

Introduction

Chain-of-Thought prompting and computational cost

CoT success prediction goal

Dataset and probing classifier setup

Mid-CoT behavior and representation similarity

Early stopping experiments

Conclusion

Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completion Anum Afzal Technical University of Munich [anum.afzal@tum.de](anum.afzal@tum.de) Gal Chechik Nvidia Research & Bar-Ilan University [gchechik@nvidia.com](gchechik@nvidia.com) Florian Matthes Technical University of Munich [matthes@tum.de](matthes@tum.de) Yftah Ziser Nvidia Research [yziser@nvidia.com](yziser@nvidia.com) Abstract We investigate whether the success of a zeroshot Chain-of-Thought (CoT) process can be predicted before completion. We discover that a probing classifier, based on LLM representations, performs well even before a single token is generated, suggesting that crucial information about the reasoning process is already present in the initial steps representations. In contrast, a strong BERT-based baseline, which relies solely on the generated tokens, performs worse—likely because it depends on shallow linguistic cues rather than deeper reasoning dynamics. Surprisingly, using later reasoning steps does not always improve classification. When additional context is unhelpful, earlier representations resemble later ones more, suggesting LLMs encode key information early. This implies reasoning can often stop early without loss. To test this, we conduct early stopping experiments, showing that truncating CoT reasoning still improves performance over not using CoT at all, though a gap remains compared to full reasoning. However, approaches like supervised learning or reinforcement learning designed to shorten CoT chains could leverage our classifier’s guidance to identify when early stopping is effective. Our findings provide insights that may support such methods, helping to optimize CoT’s efficiency while preserving its benefits.1 1 Introduction Chain-of-Thought (CoT) prompting (Wei et al., 2023) enhances the capability of large language models (LLMs) to perform multi-step reasoning. It explicitly guides the LLM in creating intermediate explanations to solve a problem, offering a sequence of reasoning steps while responding to a prompt. Given its effectiveness, CoT has found success in mathematical reasoning (Zheng et al., 2023), 1 Code and data is available at [github.com/anum94/CoTpred](github.com/anum94/CoTpred). Figure 1: Illustration of our approach. The LLM generates intermediate reasoning steps in a Chain-of-Thought sequence. At step i, we use its internal representations to predict whether the CoT process will succeed. The snowflake (❄) indicates frozen parameters, while the flame (\) indicates trainable parameters. medical applications (Liu et al., 2024a), faithfulness evaluation (Xu et al., 2024b), and multimodal models (Wang et al., 2024 ; Kumari et al., 2024 ; Byun et al., 2024) . While CoT reasoning has been shown to improve performance across many tasks, it is computationally expensive, as it requires decomposing complex problems into a series of intermediate steps, each demanding its own processing. This raises two intriguing questions: a) Do LLMs implicitly "know" whether they will arrive at a correct answer before completing their reasoning? and b) If progressing past the initial steps doesn’t improve this knowledge, does this indicate that the LLM has completed its calculation? Given the high computational cost of CoT, understanding 12791 Findings of the Association for Computational Linguistics: ACL 2025 , pages 12791–12806 July 27-August 1, 2025 ©2025 Association for Computational Linguistics when and how LLMs "know" their answer could enable more efficient and targeted reasoning strategies. Developing a method to assess whether CoT will lead to a correct conclusion could optimize resource allocation—stopping reasoning early when the outcome is clear or dedicating more steps when uncertainty remains. Furthermore, this knowledge could inform annotation efforts to support CoTspecific fine-tuning. To explore these questions, we create a CoT suc

FAQs

Can zero-shot chain-of-thought success be predicted before completion?

Yes. The probing classifier predicts whether CoT will succeed even before any token is generated, indicating success-relevant information exists early in LLM representations.

Why does the BERT-based baseline perform worse?

It relies only on generated tokens and shallow linguistic cues rather than deeper reasoning dynamics, whereas the LLM representations contain richer information about intermediate calculations.

Does using later reasoning steps always improve prediction accuracy?

No. In some cases, providing later CoT steps does not significantly improve classification accuracy, and representation similarity suggests that key information may already appear early.

Sep 4, 2026

Upload by Stanford