X-LOGSMASK: EXPAND TRANSFORMER FOR GRAPH-STRUCTURED DATA - A PREPRINT

Sep 21, 2026

Transformers are general-purpose architectures, but their all-to-all self-attention poorly matches graph data, whose relations are sparse, structured, and multi-scale. Existing Graph Transformers rely on structural encodings, hybrid message-passing, or learned attention constraints, often adding complexity and limiting interpretability. X-LogSMask introduces an explainable multi-head logarithmic structural mask that injects symmetrically normalized topology into attention logits.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Motivation for graph-aware attention

Related work on Graph Transformers

Method

X-LogSMask design and injection into attention

Head-wise logarithmic structural radii

Experiments

Benchmarks and performance results

arXiv :2607 .0 1553v 1 [ cs .LG] 2 Jul 2026 X-LOGSMASK: EXPAND TRANSFORMER FOR GRAPH-STRUCTURED DATA A PREPRINT Leyan Li* Rennong Yang Zhenxing Zhang Liping Hu July 3, 2026 ABSTRACT Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale. Existing Graph Transformers address this mismatch through structural encodings, hybrid message-passing modules or learned attention constraints, often introducing additional complexity and limited interpretability. Here we introduce X-LogSMask, an explainable multi-head logarithmic structural mask that injects symmetrically normalized graph topology directly into attention logits. The logarithmic transform converts structural connectivity into a topology-aware gating signal, suppressing unsupported node interactions while preserving feature-dependent attention. By assigning different powers of the normalized adjacency matrix to different attention heads, X-LogSMask gives each head a defined structural radius and supports multi-hop information propagation within a single layer. We further show that a standard Transformer encoder can be interpreted as one-step message passing on a complete graph, motivating X-LogSMask as a topology-constrained alternative to unrestricted self-attention. Across 20 node-, edge- and graph-level benchmarks, Transformers equipped with X-LogSMask achieve state-of-the-art performance on 13 datasets and remain competitive in a lightweight onelayer configuration. These results show that simple, interpretable structural masks can make selfattention an effective graph-learning operator without changing the Transformer architecture. The code is available at [https://github.com/LiLeyan-0120/X-LogSMask](https://github.com/LiLeyan-0120/X-LogSMask). Keywords Graph transformer, graph neural network, explainable learning, logarithmic structural mask, multi-hop attention 1 Introduction Graphs provide a natural representation for relational systems, including citation networks, social networks, molecules, biological systems and transportation networks. Unlike sequences or images, graphs are defined by irregular neighborhoods, sparse connectivity and dependencies that may span multiple topological scales. Graph neural networks (GNNs) address this structure by propagating information along observed edges, thereby aligning model computation with graph topology [1, 2] . By contrast, the standard Transformer was developed around self-attention, which permits every token to interact with every other token [3] . This design gives Transformers strong representational flexibility, but it also creates a mismatch when the input is a graph. Transformers have been successfully extended beyond language to vision, temporal modelling and other structured domains [4–8] . Their success suggests that self-attention can serve as a general-purpose information aggregation operator when equipped with an appropriate inductive bias. In sequential data, this role is played by positional encodings, which supply the order information absent from permutation-invariant attention. For graph data, however, the required inductive bias is not a one-dimensional position but a sparse, relational and multi-hop topology. Without such a bias, all-to-all self-attention can mix topologically unrelated nodes and weaken the structural constraints that make message passing effective. ∗ E-mail: li [leyan@163.com](leyan@163.com); [rangrn6907@163.com](rangrn6907@163.com); [zzx2207621676@hotmail.com](zzx2207621676@hotmail.com); Hlp [afeu@163.com](afeu@163.com) RNN for 1D CNN for 2D • VIT [Dosovitskiy et al.] • et al. Figure 1: Transformer for all kinds of data. Recent Graph Transformer models have therefore sought to inject graph structure into attention. One line of work incorporates attention mechanisms into GNN architectures, as in Graph Attention Networks [9] .

FAQs

What problem does X-LogSMask address in Transformers for graph data?

It addresses the mismatch between Transformers’ unrestricted all-to-all self-attention and graphs’ sparse, structured, multi-scale topology by injecting topology information into attention.

How does X-LogSMask modify the attention mechanism?

It constructs a symmetrically normalized adjacency matrix with self-loops and injects a logarithmic, topology-aware gating signal additively into the attention logits.

What is the role of multi-head logarithmic structural masks in X-LogSMask?

Different attention heads use different powers of the normalized adjacency matrix, giving each head an explicit structural radius and enabling multi-hop information propagation within a single layer.

Sep 21, 2026

Uploaded by Ava Thompson