Silent Failures in Quantized LLM Reasoning A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

Jul 16, 2026

Upload by Aria

Post-training quantization can silently change how large language models reason even when benchmark accuracy remains stable. The study validates a six-category failure taxonomy (Cohen’s κ=0.906) and annotates 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B–14B) across FP32, FP16, and NF4, spanning four reasoning benchmarks. Hollow Convergence shifts size-dependently under NF4, while some benchmarks show immunity. Shortcut Collapse rises and Confidence Snowballing collapses under NF4, and Hollow Convergence cannot be detected from surface features (best F1=0.53).

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

Introduction

Contributions

Related Work

Post-Training quantization

Chain-of-Thought Faithfulness

Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts Renuka Oladri Science Academy University of Maryland College Park, MD, USA[roladri@umd.edu](roladri@umd.edu) Mohan Vamsi Varadaraju Priya Science Academy University of Maryland College Park, MD, USA [mvamsivp@umd.edu](mvamsivp@umd.edu) Jerry Wu Electrical and Computer Engineering University of Maryland College Park, MD, USA [jerrywu@umd.edu](jerrywu@umd.edu) Abstract— We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved. Using a six-category failure taxonomy validated by two independent human annotators (Cohen's κ = 0.906), we classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B–14B parameters) across three quantization precisions (FP32, FP16, NF4) and four reasoning benchmarks. We find that while accuracy is robust across precisions (maximum 3.1 pp drop), Hollow Convergence (correct answers reached through incomplete or unverifiable reasoning) shows a significant size-dependent shift under NF4, dropping sharply for the two smallest models tested but remaining invariant for models at 12B parameters and above. This effect is also benchmark-specific: GSM8K is categorically immune while LogiQA and ARC-Challenge show the largest shifts. Furthermore, under NF4, Shortcut Collapse rises from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B while Confidence Snowballing collapses from 15.8% to near zero, a qualitative shift invisible to accuracy metrics. Finally, we show Hollow Convergence cannot be reliably detected from surfacelevel text features (best F1 = 0.53), establishing it as a deployment-relevant failure mode that standard evaluation pipelines cannot catch. Keywords—quantization, large language models, chain-ofthought, reasoning failure, hollow convergence, NF4. I. INTRODUCTION Post-training quantization has emerged as a standard technique for deploying large language models under memory and latency constraints [1][2][3] . This pressure is intensifying as deployment moves toward resourceconstrained and on-device settings, which has driven specialized hardware accelerators for both quantizationaware training [20] and layer-wise post-training quantization [21], the latter explicitly treating attention and feed-forward sublayers as having different sensitivity to precision reduction, a distinction we return to when interpreting our own results. The difficulty these accelerators are built to manage is well documented at the source: activation outliers in LLM intermediate representations remain a primary driver of quantization error, and resolving them is still an open problem [22]. Despite this difficulty, practitioners widely rely on accuracy metrics to validate that quantized models are safe for deployment, a practice reinforced by recent comprehensive evaluations showing that combining adapterbased fine-tuning with post-training quantization preserves downstream task performance [23]. This is underpinned by a robust empirical finding: quantized models largely match their full-precision counterparts on standard benchmarks [4] . We challenge a hidden assumption in this practice: that accuracy is a sufficient proxy for reasoning quality. A model that produces the correct answer through flawed, incomplete, or unverifiable reasoning is not reasoning correctly. We call this failure mode Hollow Convergence: outputs that are correct by the answer key but whose chain-of-thought either skips essential steps, states conclusions without derivation, orrestates the question without solving it. Prior work on chain-of-thought faithfulness demonstrates that reasoning chains frequently do not reflect models’ true internal processes [5][6] . Models can produce plausiblesounding reasoning causally disconnected from their answer [7]. What is not known is whether quantization systematically worsens this p

FAQs

What failure mode shifts occur under NF4 that remain invisible to accuracy metrics?

Under NF4, Shortcut Collapse increases from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B, while Confidence Snowballing drops from 15.8% to near zero. These changes reflect qualitative shifts in failure behavior without corresponding accuracy differences.

Jul 16, 2026

Upload by Aria