Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

Jul 16, 2026

Self-refinement in large language models often fails to improve few-shot inductive reasoning, especially when models can directly patch outputs after feedback rather than consolidating the underlying rule. Hourglass reasoning introduces structurally enforced isolation between reasoning stages via a frozen meta-constructor that builds an encoder–decoder symbolic bottleneck. Only the compressed symbolic state crosses stages, so refinement remains anchored to the inferred rule. Experiments across visual abstraction, hardware synthesis, and textual rule induction show large accuracy and synthesis gains and ablations confirm the effect comes from stage isolation and induction quality, not prompt wording.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages27

Reading Time1 h 8 min

Introduction

Bottleneck motivation and information flow

Hourglass reasoning overview

Evaluation across benchmarks

Ablation insights

arXiv :2607 . 1 1696v 1 [ cs .AI] 13 Jul 2026 Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction Huan Zhu∗ Peking University ABSTRACT Self-refinement often fails to strengthen few-shot inductive reasoning in large language models. Prompting a model to explicitly state its inferred rule does little on its own. What actually matters is a structurally enforced isolation between reasoning stages, so that information can only pass between them as a compressed symbolic state. We introduce Hourglass reasoning, which enforces strict context isolation between reasoning stages. The frozen LLM acts as a meta-constructor, building for each task a symbolic encoder–decoder: an Induction module compresses the support examples into a schema 􀁱 (encoder) and a transient scaffold 􀁉 ; a Deduction module derives rule 􀀩 (decoder) from these and discards 􀁉 ; an Implementer compiles (􀁱, 􀀩) into artifacts; an error-driven Refiner revises (􀁱, 􀀩) and regenerates artifacts from scratch. Only (􀁱, 􀀩) crosses stage boundaries, so all refinement stays anchored to the rule. We evaluate Hourglass across three benchmarks spanning visual abstraction, hardware synthesis, and textual rule induction, using GPT-5.5 and Gemini 3.1 Pro. On ARC-AGI-2, it raises best-of-5 accuracy by up to 14 points over an iterative-refinement baseline. On ChipBench, it nearly doubles Verilog synthesis accuracy with GPT-5.5, from 31% to 58% . BBEH-Linguini draws on puzzles from the International Linguistics Olympiad, a setting where prior work has shown that explicit verbalization can hurt performance. Hourglass mitigates this tendency, and on Gemini 3.1 Pro, it reverses the effect entirely. Ablations confirm that these gains come from the isolation between stages and the quality of the initial induction, not from prompt wording or the particular symbolic form used. It is how information flows through the reasoning process, rather than the language used to express it, that drives inductive reasoning in frozen LLMs. 1 Introduction Humans can extract abstract rules from only a handful of examples. In artificial intelligence, this capability is rigorously tested by benchmarks like ARC-AGI-2 (Chollet, 2019), where each puzzle is governed by a single, precise transformation rule that must be inferred from few-shot demonstrations. Despite impressive ∗ Code and prompts: [https://github.com/ZhuHuan09/hourglass-reasoning](https://github.com/ZhuHuan09/hourglass-reasoning) performance on many natural-language tasks, current frontier LLMs still struggle with such rule-centric induction. Large language models have demonstrated remarkable capabilities across a broad spectrum, extending from natural language processing and code generation to hardware description language synthesis (Liu, Y., et al., 2025) and abstract spatial reasoning on benchmarks such as ARC-AGI-2 (Franzen et al., 2025) . However, these models remain prone to shortcut learning (Geirhos et al., 2019): instead of abstracting the latent rule, a model exploits superficial regularities in the support examples. When execution feedback is available, this often manifests as patchwork logic: hardcoded if-else branches keyed to specific coordinates, example indices, or local artifacts. Such patches force support examples to pass but fail on out-of-distribution queries (Moskvichev et al., 2023; Mitchell et al., 2023) . Moreover, a growing body of recent work demonstrates that the intrinsic self-correction capabilities of monolithic LLMs are brittle and frequently degrade performance in the absence of external structural guidance (Tsui, 2025; Sanz-Guerrero & Von Der Wense, 2025) . These pathologies are partly rooted in how information flows through dense, unrestricted context windows. When raw examples, current artifacts, error feedback, and repair instructions coexist without structured partition, the model tends to anchor on low-level perceptual details rather than generalizing to an abstract

FAQs

What problem does Hourglass reasoning target in few-shot inductive reasoning?

It targets failures where self-refinement does not strengthen rule-centric induction in frozen LLMs. Models may instead exploit superficial regularities and patch outputs directly when feedback is available.

How does Hourglass reasoning enforce correct information flow across stages?

It isolates reasoning stages so that only a compressed symbolic state (an encoder schema and a decoder rule) crosses stage boundaries. Intermediate traces, including transient scaffolds, are discarded to prevent instance-specific leakage.

What experimental benefits does Hourglass achieve and how are they validated?

Across three benchmarks, Hourglass improves performance over a context-reset self-refinement baseline, including higher ARC accuracy and better Verilog synthesis. Ablations attribute gains to stage isolation and the quality of the initial induction rather than prompt wording or symbolic form.

Jul 16, 2026

Upload by Valentina