THINK-ON-GRAPH: DEEP AND RESPONSIBLE REASONING OF LARGE LANGUAGE MODEL ON KNOWLEDGE GRAPH

Oct 12, 2026

Upload by Hazel

Large language models (LLMs) face hallucination and limited reliability when deep, responsible reasoning is required, especially for multi-hop questions needing specialized, up-to-date knowledge. The paper introduces a tight-coupling LLM-KG paradigm, “LLM ⊗ KG,” and implements it with Think-on-Graph (ToG). ToG uses an LLM agent that iteratively beam searches on knowledge graphs to discover promising reasoning paths and return likely results. Experiments show ToG improves deep reasoning, enables traceability and correctability, and offers a plug-and-play framework without extra training, achieving overall SOTA on 6/9 datasets.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages31

Reading Time11 min

AccessFree to download

Abstract

Introduction

Limitations of LLM-only reasoning

LLM ⊕ KG paradigm and its limitations

Proposed tight-coupling LLM ⊗ KG and Think-on-Graph

arXiv :2307 .07697v6 [ cs .CL] 24 Mar 2024 THINK-ON-GRAPH: DEEP AND RESPONSIBLE REASONING OF LARGE LANGUAGE MODEL ON KNOWLEDGE GRAPH Jiashuo Sun21∗†Chengjin Xu 1∗Lumingyuan Tang31∗†Saizhuo Wang41∗† Chen Lin2 Yeyun Gong6 Lionel M. Ni5 Heung-Yeung Shum 14 Jian Guo 15‡ 1IDEA Research, International Digital Economy Academy 2Xiamen University 3University of Southern California 4The Hong Kong University of Science and Technology 5The Hong Kong University of Science and Technology (Guangzhou) 6Microsoft Research Asia ABSTRACT Although large language models (LLMs) have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning. These issues could be partially addressed by introducing external knowledge graphs (KG) in LLM reasoning. In this paper, we propose a new LLM-KG integrating paradigm “LLM ⊗ KG” which treats the LLM as an agent to interactively explore related entities and relations on KGsand perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages ofToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plugand-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training. Our code is publicly available at [https://github.com/IDEA-FinAI/ToG](https://github.com/IDEA-FinAI/ToG). 1 INTRODUCTION Large language models (LLMs) (Ouyang et al., 2022; OpenAI, 2023; Thoppilan et al., 2022; Brown et al., 2020a; Chowdhery et al., 2022; Touvron et al., 2023) have demonstrated remarkable performance across various natural language processing tasks. These models capitalize on pre-training techniques applied to vast text corpora to generate responses that are coherent and contextually appropriate. Despite their impressive performance, LLMs have substantial limitations when facing complex knowledge reasoning tasks (Petroni et al., 2021; Talmor et al., 2019; Talmor & Berant, 2018; Zhang et al., 2023) that require deep and responsible reasoning. Firstly, LLMs usually fail to provide accurate answers to questions requiring specialized knowledge beyond what was included in the pre-training phase (out-of-date knowledge in Figure 1a), or to questions requiring long logic chain and multi-hop knowledge reasoning. Secondly, LLMs lack responsibility, explainability and transparency, ∗Equal contribution. †Work done during internship at IDEA Research.‡Corresponding author. (a) |
LLM ⊕ KG | | | --- | --- | | (Prompt): Please generate a SPARQL query for this question.
Response: SELECT ?country ?party WHERE {?canberra dbprop:isLocatedIn ?country .
?country dbprop:majorityParty ?party .
} ~~ ~~ | | Australia Labor Party majorityParty [not exist] (b) Prompt Response: Sorry, based on my query result from the knowledge base, I cannot answer your question since I do not have enough information. “Think” “Think” | Canberra capitofal
| |

| |
| has been leader of the Australian Labor Party (ALP) since 2019 and | | --- | --- | --- | --- | --- | --- | | | | | | | | (c) Figure 1: Representat

FAQs

What problem does the paper address with large language models?

The paper targets hallucinations and unreliable deep, responsible reasoning, especially for tasks requiring specialized, multi-hop knowledge.

How does the proposed “LLM ⊗ KG” paradigm differ from “LLM ⊕ KG”?

In “LLM ⊕ KG,” the LLM mainly translates the question for KG search and reasoning without directly participating in graph reasoning. In “LLM ⊗ KG,” KGs and LLMs work in tandem, with the LLM agent exploring reasoning paths on the KG.

What is Think-on-Graph (ToG) and how does it produce answers?

Think-on-Graph is implemented by an LLM agent that iteratively performs beam search over the knowledge graph to discover the most promising reasoning paths and then returns the most likely reasoning results.

Oct 12, 2026

Upload by Hazel