Do Large Language Models Play Six Degrees of Separation? - Measuring Topological Compression in Long-Context Manifolds - Abstract

Sep 15, 2026

Uploaded by Maeve

Large Language Models (LLMs) exhibit strong multi-hop reasoning over long contexts, yet their internal mechanisms for bridging distant concepts are not well understood. This work bypasses attention-weight interpretability and instead analyzes the hidden-state manifold geometry. The study sparsifies long-context similarity into unweighted graphs to reveal Small-World organization and a topological phase transition across architectures and layer depth. It also evaluates zero-shot hallucination detection in Retrieval-Augmented Generation using the RAGognize dataset.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Abstract

1 Introduction

Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds Md. Faiyaz Abdullah Sayeedi BRAC University, Bangladesh [msayeedi212049@bscse.uiu.ac.bd](msayeedi212049@bscse.uiu.ac.bd) arXiv : 2608 . 17950v2 [cs .CL] 2 1 Aug 2026 Abstract Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, wetrace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the “Six Degrees of Separation”limit (≤ 6 semantic hops) . Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (≈ 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability. 1 Introduction “A fascinating game grew out of this discussion. One of us suggested performing the following experiment to prove that the population of the Earth is closer together now than they have ever been before. We should select any Figure 1: Illustration of Topological Compression. A 3D PCA projection of an LLM’s deep latent manifold. The highlighted Small-World pathway bridges fundamentally disjoint anchors, Mosquito (ecology) and Jailbreak (cybersecurity), across a dense cloud of unrelated context tokens (gray) . The model compresses this massive conceptual distance into just 5 hops by leveraging polysemous intermediate concepts (e.g., Virus) . person from the 1.5 billion inhabitants of the Earth; anyone, anywhere at all. He bet us that, using no more than five individuals, one of whom is a personal acquaintance, he could contact the selected individual using nothing except the network of personal acquaintances.” —Frigyes Karinthy, Chains (1929) Nearly a century after Frigyes Karinthy first proposed the “six degrees of separation,” the concept that massive, complex networks can be navigated through shockingly short paths, a phenomenon later formalized as Small-World networks (Watts and Strogatz, 1998), has been proven across human sociology (White and Houseman, 2002), biological neural networks (Bassett and Bullmore, 2006), and the global internet (Jin and Bestavros, 2006) . Today, a new class of massive, complex networks governs the frontier of artificial intelli- gence: Large Language Models. As the context windows of these models expand from thousands to millions of tokens, a fundamental question emerges regarding their internal cognitive architecture: How do Transformer-based models successfully route, bridge, and reason across physically distant and semantically disjoint concepts? Do they possess their own “Small-World” topology? Understanding the structural geometry of LLM reasoning is critical. While modern LLMs exhibit remarkable capabilities in long-context retrieval, in-context learning, and multi-step reasoning, the topological mechanisms enabling

FAQs

Why do traditional attention-based interpretability methods fail for long-context semantic proximity?

Attention matrices can collapse by routing weights to sink tokens rather than forming continuous semantic bridges. As a result, attention does not reliably reflect true topological proximity between abstract concepts.

What is the main method used to study long-context reasoning in this paper?

The paper bypasses attention weights and measures the dynamic geometry of the hidden-state manifold. It uses a Semantic Anchor methodology to track how LLMs compress semantic gaps across layers.

What topological behavior do deep LLM layers show compared with early layers?

Deep reasoning layers abruptly compress large conceptual distances into highly navigable Small-World pathways. Early syntactic layers remain fractured, while compression is bounded by the “Six Degrees of Separation” limit (≤ 6 semantic hops).

How is the framework applied to hallucination detection in Retrieval-Augmented Generation (RAG)?

The approach is used for zero-shot hallucination detection with the RAGognize dataset. Factually grounded generations preserve topological structure with about 3 semantic hops, while hallucinations cause severe topological collapse.

Sep 15, 2026

Uploaded by Maeve