[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118992-en":3,"doc-seo-118992-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118992,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","How does over-squashing affect the power of GNNs - Paper","Graph Neural Networks (GNNs) power machine learning on graph-structured data, with Message Passing Neural Networks (MPNNs) forming the dominant paradigm via information exchange between adjacent nodes. While MPNN expressivity is central, prior work often assumes uninformative node features. This study delivers a rigorous capacity-aware analysis by quantifying pairwise interactions permitted by an MPNN and characterizing over-squashing, where many messages are compressed into fixed-size vectors.","arXiv :2306 .03589v3 [ cs .LG] 12 Feb 2024  \nHow does over-squashing affect the power of GNNs?  \nFrancesco Di Giovanni∗ [francesco. di.giovanni@cs. ox. ac.uk](francesco. di.giovanni@cs. ox. ac.uk)  \nUniversity of Oxford  \nT. Konstantin Rusch∗ [tkrusch@mit. edu](tkrusch@mit. edu)[ ](tkrusch@mit. edu)Massachusetts Institute of Technology  \nMichael M. Bronstein  \nUniversity of Oxford  \nAndreea Deac  \nUniversité de Montréal  \nMarc Lackenby  \nUniversity of Oxford  \nSiddhartha Mishra  \nETH Zürich  \nPetar Veličković  \nGoogle DeepMind  \nReviewed on OpenReview: [https: // openreview. net/ forum? id= KJRoQvRWNs](https: // openreview. net/ forum? id= KJRoQvRWNs)  \nAbstract  \nGraph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs) .  \nWhile understanding the expressive power of MPNNs is a key question, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. Our theory also holds for geometric graphs and hence extends to equivariant MPNNs on point clouds. We validate our analysis through extensive controlled experiments and ablation studies.  \n1 Introduction  \nGraphs describe the relational structure for a large variety of natural and artificial systems, making learning on graphs imperative in many contexts (Veličković, 2023 ; DeZoort et al. , 2023 ; Williamson, 2023) . To this end, Graph Neural Networks (GNNs) (Gori et al. , 2005 ; Scarselli et al. , 2008) have emerged as a widely popular framework for graph machine learning, with plentiful success stories in science (Bapst et al. , 2020 ;  \n∗ Equal contribution.  \nFigure 1: We study the power of MPNNs in terms of the mixing they induce among features and show that this is affected by the model (via norm of the weights and depth) and the graph topology (via commute times) . For the given graph, the MPNN learns stronger mixing (tight springs) for nodes v, u and u, w since their commute time is small, while nodes u, q and u, z, with high commute-time, have weak mixing (loose springs) . We characterize over-squashing as the inverse of the mixing induced by an MPNN and hence relate it to its power. In fact, the MPNN might require an impractical depth to solve tasks on the given graph that depend on high-mixing of features assigned to u, z.  \nDavies et al. , 2021 ; Blundell et al. , 2021 ; Stokes et al. , 2020 ; Liu et al. , 2023) and technology (Monti et al. , 2019 ; Mirhoseini et al. , 2021 ; Derrow-Pinion et al. , 2021) .  \nGiven an underlying graph and features, defined on its nodes (and edges), as inputs, a GNN learns parametric functions from data. Due to the ubiquity of GNNs, characterizing their expressive power, i.e., which class of functions a GNN is able to learn, is a problem of great interest. In this context, most available results in literature on the universality of GNNs pertain to impractical higher-order tensors (Maron et al. , 2019 ; Keriven & Peyré , 2019) or unique node identifiers that may break the symmetries of the problem (Loukas, 2020) . In particular, ","cbCaioPIDgLkJUk0","https://ap.wps.com/l/cbCaioPIDgLkJUk0","pdf",1069308,1,36,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does the paper study about GNNs?\",\"answer\":\"The paper studies how over-squashing affects the expressive power of message passing GNNs, focusing on which function classes MPNNs can learn for a given model capacity.\"},{\"question\":\"How is over-squashing characterized in the analysis?\",\"answer\":\"Over-squashing is characterized as the inverse of the mixing induced by an MPNN, driven by how aggregated messages collapse into fixed-size vectors.\"},{\"question\":\"What determines whether an MPNN can communicate between node pairs?\",\"answer\":\"Sufficient communication between node pairs requires enough MPNN capacity, depending on graph-structure properties such as commute times.\"}]","How does over-squashing affect the power of GNNs - Paper | PDF",1785721514,91,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"how-does-over-squashing-affect-the-power-of-gnns-paper","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/how-does-over-squashing-affect-the-power-of-gnns-paper/118992/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper study about GNNs?","Question",{"text":75,"@type":76},"The paper studies how over-squashing affects the expressive power of message passing GNNs, focusing on which function classes MPNNs can learn for a given model capacity.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is over-squashing characterized in the analysis?",{"text":80,"@type":76},"Over-squashing is characterized as the inverse of the mixing induced by an MPNN, driven by how aggregated messages collapse into fixed-size vectors.",{"name":82,"@type":73,"acceptedAnswer":83},"What determines whether an MPNN can communicate between node pairs?",{"text":84,"@type":76},"Sufficient communication between node pairs requires enough MPNN capacity, depending on graph-structure properties such as commute times.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]