[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83115-en":3,"doc-seo-83115-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83115,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion","Graph denoising is a core task in graph learning and the central operation behind graph diffusion models, where attention-based architectures such as graph transformers have recently shown strong results. A theoretical gap remains about whether standard attention is the right mechanism for denoising graphs. Under a denoising objective, linear attention is suboptimal, learning only an average spectral denoising filter. Spectral Attention and its practical, permutation-equivariant realization GCA improve performance with gains tied to spectral diversity.","Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion  \nShervin Khalafi∗  \nUniversity of Pennsylvania [shervink@seas.upenn.edu](shervink@seas.upenn.edu)  \nIgor Krawczuk∗ [igor@krawczuk.eu](igor@krawczuk.eu)  \nSergio Rozada  \nKing Juan Carlos University [sergio.rozada@urjc.es](sergio.rozada@urjc.es)  \nCharilaos Kanatsoulis  \nStanford University  \n[charilaos@cs.stanford.edu](charilaos@cs.stanford.edu)  \nAntonio G. Marques  \nKing Juan Carlos University antonio.garcia.marques@urjc.es  \narXiv :2607 .06546v 1 [ cs .LG] 7 Jul 2026  \nAlejandro Ribeiro  \nUniversity of Pennsylvania  \n[aribeiro@seas.upenn.edu](aribeiro@seas.upenn.edu)  \nAbstract  \nDenoising graphs is a fundamental problem in graph learning and the core operation of graph diffusion models. Attention-based architectures like graph transformers have recently shown promise in denoising graphs. However, our principled understanding of attention-based graph denoising remains limited, making it unclear whether standard attention is the right mechanism for this task. Here we show that, under a denoising objective, linear attention is suboptimal and can only learn an average spectral denoising filter over the training distribution. This creates a fundamental limitation as graphs often vary spectrally across the distribution. To overcome this limitation, we introduce Spectral Attention, which directly utilizes the input graph spectrum and provably outperforms linear attention by a margin governed by the spectral diversity of the distribution. We then derive Graph Convolutional Attention (GCA), a practical and permutation-equivariant realization of this idea that implements spectral denoising through graph-filtered queries and keys. For stochastic block models, GCA provably matches the idealized Spectral Attention mechanism. We further show that the softmax operation, that follows the attention, provides additional denoising by approximately projecting noisy eigenvectors onto the clean eigenspace. Empirically, replacing linear attention with GCA consistently improves graph denoising and diffusion on synthetic and real datasets, with gains strongly correlated with spectral diversity. In DiGress, GCA matches standard graph-transformer performance without computing expensive structural features, and when combined with the recently proposed PEARL positional encodings, avoids explicit eigendecomposition computations resulting in faster inference without degrading quality. The code can be found here:  \n[github.com/shervinkhalafi/graph_conv_att](github.com/shervinkhalafi/graph_conv_att)  \n1 Introduction  \nDenoising is a fundamental problem in machine learning [29], both in its own right and as the central  \nlearning primitive behind denoising diffusion models [39, 40] . Graph denoising introduces a unique ∗Equal contribution.  \nPreprint.  \nchallenge: the object to be denoised is not merely a signal on a known domain, but the domain itself. In image and audio denoising, the underlying grid or temporal axis is fixed, so models can exploit geometry through convolutions [28] . In graphs, by contrast, the geometry and spectral frequency is corrupted and must itself be recovered. Classical approaches address graph denoising through regularized optimization [46, 38], but the parametric perspective has received comparatively little attention. Interest has grown recently, driven by graph diffusion models [14, 44], where the dominant approach is graph transformers that take the eigenvectors of the graph, and optionally additional structural features, as input. Yet a theoretical understanding of how such parametric models learn to denoise graphs remains limited. In this work, we aim to provide insight into the mechanisms underlying parametric graph denoising through theoretical analysis.  \nWe proceed in four steps. First, we (i) analyze linear attention, the outer product of linear query-key projections, as it underlies every attention-based graph denoiser. We show i","cbCaiqWxTtM0YNys","https://ap.wps.com/l/cbCaiqWxTtM0YNys","pdf",5637086,1,34,"English","en",105,"# Introduction\n## Linear attention analysis\n## Spectral attention\n## Graph convolutional attention (GCA)\n## Softmax as eigenvector denoising\n# Experiments","[{\"question\":\"Why is linear attention considered suboptimal for graph denoising?\",\"answer\":\"With a denoising objective, linear attention can only learn an average spectral denoising filter over the training distribution, which becomes limited when graphs vary spectrally across that distribution.\"},{\"question\":\"What is Spectral Attention and what advantage does it provide?\",\"answer\":\"Spectral Attention is an attention class that depends directly on input graph eigenvalues. Its loss is strictly smaller than linear attention, with the improvement governed by the spectral diversity of the distribution.\"},{\"question\":\"How does Graph Convolutional Attention (GCA) make Spectral Attention practical?\",\"answer\":\"GCA provides a permutation-equivariant realization where the attention pattern is a pointwise function of eigenvalues, implemented as a graph-filtered mechanism using filtered queries and keys. It can match idealized Spectral Attention under stochastic block model assumptions.\"}]",1784185374,86,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"graph-convolutional-attention-a-spectral-perspective-on-graph-denoising-and-diffusion","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/graph-convolutional-attention-a-spectral-perspective-on-graph-denoising-and-diffusion/83115/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is linear attention considered suboptimal for graph denoising?","Question",{"text":75,"@type":76},"With a denoising objective, linear attention can only learn an average spectral denoising filter over the training distribution, which becomes limited when graphs vary spectrally across that distribution.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is Spectral Attention and what advantage does it provide?",{"text":80,"@type":76},"Spectral Attention is an attention class that depends directly on input graph eigenvalues. Its loss is strictly smaller than linear attention, with the improvement governed by the spectral diversity of the distribution.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Graph Convolutional Attention (GCA) make Spectral Attention practical?",{"text":84,"@type":76},"GCA provides a permutation-equivariant realization where the attention pattern is a pointwise function of eigenvalues, implemented as a graph-filtered mechanism using filtered queries and keys. It can match idealized Spectral Attention under stochastic block model assumptions.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]