WavePhaseNet - A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)

Sep 15, 2026

Uploaded by Himbo

This paper reinterprets Transformer/attention mechanisms in large language models through measure theory and frequency analysis, establishing hallucination as an inherent structural limitation. It shows that embedding spaces behave like conditional expectations over a σ-algebra, and that mismatch with the semantic truth set breaks logical consistency. WavePhaseNet constructs a Semantic Conceptual Hierarchy Structure (SCHS) using DFT, separating meanings into frequency bands and enabling rigorous semantic manipulation. A spectral analysis suggests a feasible lower-dimensional “complete representation” bound, and a cohomology-inspired regularizer enforces global consistency via harmonic projection.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

AccessFree to download

Abstract

Introduction

Autoregressive Models and the Embedding Space as a σ-algebra

Measure-Theoretic Formulation of the Autoregressive Model

WavePhaseNet: A DFT-Based Method for Constructing Semantic arXiv :2602 . 144 19v2 [ cs .CL] 16 Jul 2026 Conceptual Hierarchy Structures (SCHS) Kiyotaka Kasubuchi [affiliation] Kazuo Fukiya [affiliation] April 12, 2026 Abstract In this paper, we reformulate the Transformer/Attention mechanisms in Large Language Models (LLMs) from the perspectives of measure theory and frequency analysis, and theoretically clarify the inevitability of hallucination as a structural limitation. In particular, we focus on the fact that the embedding space functions as a conditional expectation over a σ-algebra, and demonstrate that the failure of this framework to be isomorphic to the semantic truth set is the fundamental cause of the breakdown of logical consistency. Based on this observation, we propose WavePhaseNet, a method for explicitly constructing a Semantic Conceptual Hierarchy Structure (SCHS) within the embedding space using the Discrete Fourier Transform (DFT) . By applying a DFT along the sequence dimension, meaning is decomposed into frequency bands: low frequencies are progressively separated as global meaning and intent, and high frequencies as local syntax and expression. This makes it possible to carry out semantic manipulation rigorously in a diagonalized space. As a concrete example, we show that the 24,576-dimensional high-dimensional embedding space, typified by GPT-4, exhibits a 1/f spectral structure based on the self-similarity of language and Zipf’slaw. Through cumulative energy analysis, we theoretically derive that the lower bound of “complete representation” is approximately 3,000 dimensions. This demonstrates that dimensionality reduction from 24,576 to 3,000 dimensions is possible while preserving meaning and intent, thereby enabling rigorous reasoning while suppressing hallucination. Finally, since the reduced low-dimensional embedding space is constructed by combining a cohomological regularization principle over overlapping local windows, a graph structure can be introduced over the window covering, and a cochain complex can be defined, so that inconsistencies among local inferences can be quantified as a loss based on coboundaries. By applying harmonic projection based on Hodge theory to extract a maximally consistent global representation, this positions cohomology not as a purely topological invariant, but as a computable regularization principle for controlling semantic consistency. The knowledge-reasoning process thereby maintains validity and rigor through the gluing of knowledge fragments. 1 Introduction Transformer-based LLMs realize natural language generation as autoregressive (AR) models by sequentially approximating the conditional probability distribution of a token sequence. While this framework demonstrates extremely high performance in grammatical consistency and expressive diversity, it inevitably produces erroneous generations—so-called hallucinations (false outputs)—in scenarios that demand a high degree of logical reasoning or factual consistency. Reasoning in LLMs relies primarily on locally computed intermediate representations, and global semantic consistency is guaranteed only implicitly. Although the self-attention mechanism can handle long-range dependencies, it lacks any mechanism to directly suppress the semantic inconsistencies that arise between overlapping local contexts. The purpose of this paper is to position hallucination not as a mere consequence of insufficient training or data defects, but as an inevitable phenomenon arising from the mathematical structure of the model itself, and to provide, as its solution, a reconstruction principle for the embedding space. Meanwhile, spectral analysis of token-embedding sequences has suggested that global semantic structure and local contextual/syntactic structure can be separated (see the prior work FNet; Lee-Thorp et al., 2021) . In this paper, we integrate these two perspectives and propose a new inferen

FAQs

What mathematical viewpoint does the paper use to explain hallucination in LLMs?

It treats embedding space as a conditional expectation over a σ-algebra and argues that the embedding framework is not isomorphic to the semantic truth set. This fundamental mismatch leads to breakdowns in logical consistency, manifesting as hallucination.

How does WavePhaseNet use the Discrete Fourier Transform (DFT) in embedding space?

WavePhaseNet applies a DFT along the sequence dimension so meaning decomposes into frequency bands. Low frequencies are associated with global meaning and intent, while high frequencies capture local syntax and expression, enabling diagonalized semantic manipulation.

What does the paper claim about dimensionality reduction in high-dimensional embeddings?

Using cumulative energy analysis, it derives a lower-bound estimate for “complete representation” at roughly 3,000 dimensions from a 24,576-dimensional embedding setting. It claims reduction is possible while preserving meaning and intent to support more rigorous reasoning and suppress hallucination.

How is semantic consistency enforced in the proposed framework?

The reduced low-dimensional embedding is built with a cohomological regularization principle over overlapping local windows. By introducing a graph structure over the window covering and defining a cochain complex, inconsistencies among local inferences are penalized as a loss term based on coboundaries, then refined via harmonic projection from Hodge theory.

Sep 15, 2026

Uploaded by Himbo