TREETHINK: A Modular Tree Search Library for Mathematical Reasoning with LLMs

Jul 16, 2026

Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries mainly target natural-language reasoning and lack native integration with formal verifiers, while theorem proving systems often use task-specific search code. TREETHINK is an open-source modular Python library providing fully asynchronous tree search with vLLM inference, diverse node evaluators, and direct REPL integration for Lean 4, Rocq, Isabelle/HOL, and natural language verification.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

Abstract

Introduction

Background: neural theorem proving and tree search

Limitations of existing LLM tree search libraries

Contributions of TREETHINK

TREETHINK: A Modular Tree Search Library for Mathematical Reasoning with LLMs Burak S. Akbudak 1 , Zeynel A. Ulusan2 , Can S. Erer 1 , Gözde Gül ¸Sahin3 , 4 , 5 1 Computer Engineering Department, Bogazici University, Istanbul, Turkey 2 Codeway Studios 3 Friedrich-Alexander-Universität Erlangen-Nürnberg, Intelligent Language Systems 4 Computer Engineering Department, Koç University, Istanbul, Turkey 5 KUIS AI Lab, Istanbul, Turkey [https://gglab-ku.github.io/](https://gglab-ku.github.io/) arXiv :2607 . 1 1258v 1 [ cs .CL] 13 Jul 2026 Abstract Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily target natural language reasoning and do not provide native integration with formal verifiers, while theorem proving systems often rely on task-specific search implementations. We introduce TREETHINK, an open-source Python library for modular, fully asynchronous tree search in neural theorem proving. It integrates established tree search methods with vLLM-based inference pipelinesand diverse node evaluation techniques, ranging from lightweight heuristics to neural evaluators. We support Lean 4, Rocq, and Isabelle/HOL alongside natural language. It connects directly to each language’s Read-EvalPrint Loop (REPL) server for real-time verification and proof state extraction. We evaluate TREETHINK on miniF2F and MATH500, demonstrating cross-language formal proof search, natural language reasoning support, and up to 6.3 × wall-clock speedup from asynchronous execution. Source code is released under the MIT license at [https://github.com/](https://github.com/)[ ](https://github.com/)GGLAB-KU/treethink, and the library is accessible as a downloadable package at [https:](https:)//[pypi.org/project/treethink/](pypi.org/project/treethink/) . 1 Introduction Mathematical reasoning remains a fundamental challenge for artificial intelligence, spanning both informal and formal paradigms (Wang et al., 2026) . While large language models (LLMs) have achieved impressive performance in informal mathematical reasoning, they remain susceptible to logical hallucinations and non-monotonic errors (Anhet al., 2025) . This has motivated an increasing interest in formal theorem proving, where interactive theorem provers (ITPs) such as Lean (de Moura and Ullrich, 2021), Rocq (The Coq Dev Team, 2024), and Isabelle (Nipkow et al., 2002) verify mathematical statements. Neural theorem proving (NTP) Figure 1: NTP tree search process. 1. Select: search method selects a node using a search algorithm. 2. Expand: the policy LLM generates child nodes. 3. Evaluate: evaluator strategy scores the generated nodes. W stands for the value assigned to a node. 4. Verify: external systems verify the correctness of the proof. Main operations in individual sections are in red while batched processes are in blue. combines these approaches by using LLMs to propose proof steps while relying on the ITP as a strict verification environment. However, NTP remains a highly difficult problem as the action space of valid mathematical tactics is vast. To guide models into successful trajectories, recent works integrate structured search algorithms into the inference process (Li et al., 2024) . Fig. 1 illustrates a typical tree search loop: the search algorithm selects which partial proof to expand, an LLM proposes the next tactic, an evaluator scores nodes, and the proof assistant checks validity. Developing novel proof search systems often requires researchers to reimplement standard execution and verification components, resulting in substantial engineering overhead. For instance, to show the effectiveness of their proposed proof level reward evaluator, Wang et al., 2023a implement a tree search mechanism from the ground up, and Li et al., 2025 orchestrate a search system to evaluate the proposed process reward models. General LLM tree search libraries such as | | Search M

FAQs

What problem does TREETHINK address in neural theorem proving?

TREETHINK targets the challenge of systematically exploring the proof space by combining LLM-proposed proof steps with strict verification using formal theorem provers, while avoiding duplicated, task-specific engineering.

How does TREETHINK integrate with formal verifiers?

It provides unified REPL clients that connect directly to Lean 4, Rocq, and Isabelle/HOL servers for real-time verification and proof state extraction.

What performance improvements does TREETHINK demonstrate?

Experiments on miniF2F and MATH500 show cross-language formal proof search, natural-language reasoning support, and up to 6.3× wall-clock speedup using fully asynchronous execution.

Jul 16, 2026

Upload by Theodora