Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking

Jul 16, 2026

Upload by Himbo

Reasoning language models often overthink by producing long, unproductive behavior chains such as hedging, abandoning approaches, and self-contradiction, which consume tokens without improving final answers. Evidence shows that even with fixed response length, incorrect reasoning traces contain more unproductive self-reflection than correct ones. DASH identifies when self-reflection helps or harms using intermediate answer commitments inside traces, then assigns segment-level credit based on drift toward or away from correctness. On competition math benchmarks, DASH improves accuracy in high-overthinking settings and enables more productive self-correction.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Introduction

Analyzing Overthinking in Reasoning Traces

Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking Chia-Hsuan Lee, Sihui Dai, Mingyang Zhou, Isha Slavin Shi-Xiong Zhang, Sambit Sahu, William Campbell Capital One arXiv :2607 .00482v 1 [ cs .CL] 1 Jul 2026 Abstract Reasoning language models frequently overthink: generating extended chains of behaviors such as hedging, approach abandonment, and self contradiction that consume tokens without improving answers. We show that these behaviors are not merely a consequence of length; even when controlling for response length, incorrect traces exhibit higher rates of unproductive self-reflection than correct ones. Addressing this requires identifying where selfreflection helps vs hurts, but obtaining these step-level annotations is costly. We observe that intermediate answer commitments within reasoning traces can provide a cheap proxy: by comparing each final answer candidate in the trace to the ground truth, we can determine whether subsequent reflection is productive without any additional supervision. Building on this insight, we propose DASH (Drift Aware advantage SHaping), which assigns segmentlevel credit based on whether each reasoning segment leads toward or away from correctness. On competition-level math benchmarks, DASH achieves the highest accuracy where overthinking is prevalent (AIME25: 50.8% vs. 45.4% GRPO) while reducing overthinking behaviorsand achieving more productive self-correction than baselines. 1 Introduction Reasoning-focused language models, such as DeepSeek-R1 (DeepSeek-AI, 2025), achieve strong performance through extended chains of thought. However, longer reasoning does not always help: models frequently exhibit overthinking behaviors such as hedging, re-verifying, or switching approaches (Wang et al., 2025b ; Peng et al., 2025) which can lead the model to an incorrect final answer. This motivates a natural question: can we train models to retain productive self-reflection while surpressing unproductive self-reflection? A signif- icant challenge is cost: identifying whether selfreflection is helpful at each reasoning step would require step-wise labels via a process reward model (Lightman et al., 2024 ; Wang et al., 2024), LLMas-a-judge, or manual annotation (Lightman et al., 2024) . In this work, we propose a cheaper alternative. Our key observation is that reasoning models can commit to intermediate answers within their thinking traces–for example, writing "the answer is X" or boxing a result before continuing to reason. These commitments provide verifiable demonstrations of productive and unproductive self-reflection: by comparing each to the ground truth, we know whether subsequent reflection improved or degraded the answer, without any external supervision. When a model reaches a correct intermediate answer and then reflects its way to an incorrect one, we have direct evidence that this self-reflection was harmful. Based on this intuition, we propose DASH (Drift-Aware advantage SHaping), which converts traces where the answer drifts from correct to incorrect intermediate examples from wasted negatives into informative training examples. Rather than assigning a single scalar advantage to the entire rollout, DASH divides each trace into segments bounded by consecutive answer checkpoints and assigns advantages based on whether each segment moves towards or away from the correct answer. A single drift trace simultaneously teaches the model to reinforce the reasoning that found the correct answer and to suppress the overthinking that abandoned it–extracting dual training signal from what GRPO would treat as a flat negative example. We complement DASH with six lightweight linguistic overthinking signals—repetition, hedging, abandonment, contradiction, recomputation, and length outlier—that characterize reasoning quality without requiring intermediate answer extraction. These signals serve as evaluation metrics to verify Figure 1: Overview of DASH. W

FAQs

What problem does the paper address in reasoning language models?

It addresses overthinking behaviors where models generate extended, token-consuming chains like hedging, approach abandonment, and self-contradiction that often lead to incorrect final answers without improving correctness.

How does DASH determine whether self-reflection is productive?

DASH uses intermediate answer commitments within reasoning traces and compares each final answer candidate in the trace to ground truth, inferring whether subsequent reflection moved the reasoning toward or away from correctness without extra step-level labels.

What training signal does DASH use instead of a single rollout-level advantage?

DASH divides each reasoning trace into segments bounded by consecutive answer checkpoints and assigns advantages per segment based on drift, so informative segments reinforce correct reasoning and harmful segments suppress overthinking.

Jul 16, 2026

Upload by Himbo