MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

Jul 16, 2026

Upload by Levi

Mathematical reasoning benchmarks often assume full information or mix reasoning with tools and retrieval. MIRA-Math isolates a narrower diagnostic ability: solving problems with a unique answer when the solver’s view is missing exactly one necessary atomic fact. The solver must request the missing fact in natural language within a strict budget and incorporate the returned fact into an exact final answer. A constrained responder shares only the dataset fact when the request matches, otherwise declines; evaluation uses deterministic generation, validation, and verification over 2,310 instances.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages62

Reading Time2 h 36 min

Introduction

Final-answer mathematical reasoning is not enough

MIRA-Math framing and protocol

Related Work

arXiv :2607 .0739 1v 1 [ cs .AI] 8 Jul 2026 MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning Charbel Al Bateh [charbel.albateh@lau.edu](charbel.albateh@lau.edu) Department of Electrical and Computer Engineering Lebanese American University Byblos, Lebanon Samer Saab [Jr.](Jr. samer.saabjr@lau.edu.lb)[ samer.saabjr@lau.edu.lb](Jr. samer.saabjr@lau.edu.lb) Department of Electrical and Computer Engineering Lebanese American University Byblos, Lebanon Abstract Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and longhorizon dialogue. We introduce MIRA-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer. A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise. Thus, instance generation, typed hint specifications, validation, and final-answer verification are deterministic, while request metrics are measured under a fixed LLM-mediated responder channel. MIRA-Math contains 2,310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. Experiments across frontier and small models show that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning. Keywords: benchmark, dataset generation, mathematical reasoning, information acquisition, clarification, partial observability, large language models 1 Introduction Large language models are increasingly evaluated on mathematical reasoning, tool use, and interactive problem solving. Yet many high-profile math benchmarks assess only the final answer under full information (Cobbe et al., 2021; Hendrycks et al., 2021; Balunovi´c et al. , 2025), while broad interactive benchmarks mix multiple sources of difficulty, including toolselection, retrieval, web navigation, external APIs, and long-context management (Mialonet al., 2023; Liu et al., 2023; Qin et al., 2023) . These settings are valuable, but they make it difficult to isolate a basic bottleneck that appears whenever a model does not initially possess all relevant information: can the model recognize the missing fact, ask for that fact precisely, and use the answer correctly? ©2026 Al Bateh and Saab. Al Bateh and Saab MIRA-Math is designed to isolate this capability. Each benchmark instance is generated from a complete mathematical state with a unique answer, but the solver model receives only a private view that is deliberately insufficient, missing exactly one necessary atomic fact. A fixed information-holder model receives only the missing fact and is constrained by a structured-output protocol: if the solver’s request semantically matches the information it holds, it returns that fact; otherwise, it returns a declination stating that it does not have the requested information. The information holder does not solve, explain, negotiate, or volunteer extra hints. The benchmark therefore measures minimal information requesting under a controlled responder channel, not open-ended multi-agent collaboration. This framing is intentionally data-centric. Following the emphasis on public benchmark infrastructu

FAQs

What problem does MIRA-Math measure in mathematical reasoning?

It measures whether a model can identify a missing atomic fact, request it precisely in natural language, and then use the received fact to produce the correct final answer.

How is the responder in MIRA-Math constrained?

A fixed constrained LLM responder sees only the single atomic fact provided by the dataset and follows a structured offer/decline protocol based on whether the solver’s request matches.

What does MIRA-Math evaluate and why are the metrics separable?

Experiments separate request success from final-answer accuracy, showing that models may ask for the right fact yet still fail the downstream computation, or fail before obtaining the canonical hint.

Jul 16, 2026

Upload by Levi