[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122443-en":3,"doc-seo-122443-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122443,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Layer-diverse Negative Sampling for Graph Neural Networks - Research Paper Summary","Graph neural networks (GNNs) excel at learning from graph-structured data, yet common message-passing models aggregate only first-order neighbours as positive samples, which can cause over-smoothing, over-squashing, and reduced expressivity. A layer-diverse negative sampling method is proposed for propagation: a sampling matrix in a determinantal point process reshapes candidate sets and selectively draws negative samples. To increase diversity across layers in multi-layer GNNs, a space-squeezing technique removes later-layer dimensions during sampling, reducing redundancy. Experiments on real-world graph datasets show improved negative-sample diversity and learning performance, with added negatives dynamically altering topology to enhance expressiveness and mitigate over-squashing.","Layer-diverse Negative Sampling for Graph Neural Networks  \nWei Duan [wei. duan@student.uts. edu. au](wei. duan@student.uts. edu. au)  \nAustralian Artificial Intelligence Institute University of Technology Sydney  \nJie Lu [jie.lu@uts. edu. au](jie.lu@uts. edu. au)  \nAustralian Artificial Intelligence Institute University of Technology Sydney  \nYu Guang Wang [yuguang.wang@sjtu. edu. cn](yuguang.wang@sjtu. edu. cn)  \nInstitute of Natural Sciences School of Mathematical Sciences Shanghai Jiao Tong University  \nJunyu Xuan [junyu.xuan@uts. edu. au](junyu.xuan@uts. edu. au)  \nAustralian Artificial Intelligence Institute University of Technology Sydney  \nReviewed on OpenReview: [https: // openreview. net/ forum? id= WOrdoKbxh6](https: // openreview. net/ forum? id= WOrdoKbxh6)  \nAbstract  \nGraph neural networks (GNNs) are a powerful solution for various structure learning applications due to their strong representation capabilities for graph data. However, traditional GNNs, relying on message-passing mechanisms that gather information exclusively from first-order neighbours (known as positive samples), can lead to issues such as over-smoothing and over-squashing. To mitigate these issues, we propose a layer-diverse negative sampling method for message-passing propagation. This method employs a sampling matrix within adeterminantal point process, which transforms the candidate set into a space and selectively samples from this space to generate negative samples. To further enhance the diversity of the negative samples during each forward pass, we develop a space-squeezing method to achieve layer-wise diversity in multi-layer GNNs. Experiments on various real-world graph datasets demonstrate the effectiveness of our approach in improving the diversity of negative samples and overall learning performance. Moreover, adding negative samples dynamically changes the graph’s topology, thus with the strong potential to improve the expressiveness of GNNs and reduce the risk of over-squashing.  \n1 Introduction  \nGraph neural networks (GNNs) have emerged as a formidable tool for various applications of structure learning, including drug discovery (Sun et al., 2020), recommendation systems (Yu & Qin, 2020), and traffic prediction (Lan et al., 2022), owing to their strong representation learning power. GNNs propagate the learning of nodes through a message-passing mechanism (Geerts et al., 2021) that conveys and aggregates information from neighbouring nodes, known as first-order neighbours. The message passing is based on the assumption that neighbours of a node have similar representations. The common practice of updating node representations solely with positive samples in most GNNs (Kipf & Welling, 2017; Xu et al., 2019; Brodyet al., 2022), can have three limitations: 1) Over-smoothing (Chen et al., 2020; Rong et al., 2020; Zhao & Akoglu, 2020), where the node representations become less distinct as the number of layers increases; 2) GNNs expressivity (Xu et al., 2019), where it becomes difficult to distinguish different graph topologies after aggregation; and 3) Over-squashing (Alon & Yahav, 2021; Topping et al., 2022; Karhadkar et al. , 2022), where bottlenecks exist and limit the information passing between weakly connected subgraphs.  \nFigure 1: Negative samples from layer-diverse DPP sampling. (a) For a given node in a graph, its first-order neighbours can be thought of as positive samples, despite the fact that these neighbours may belong to different clusters. (b) Algorithm 1 calculates the shortest path from a given node to other nodes in the graph to obtain smaller, yet more efficient candidate sets for further sampling. (c) As the candidate set is significantly larger than the number of negative samples needed, the ideal subset of negative samples is not unique. By using the layer-diverse DPP sampling method to select negative samples, it is possible to include as much information from the entire graph as possible while also reducing redundancy amon","cbCaieeDAyRMd0g1","https://ap.wps.com/l/cbCaieeDAyRMd0g1","pdf",10365295,1,28,"English","en",105,"# Introduction\n## Graph neural networks and message passing limitations\n## Negative sampling motivation and redundancy\n# Proposed method: layer-diverse negative sampling\n## DPP-based sampling matrix and candidate space\n## Space-squeezing for layer-wise diversity\n# Experiments and results","[{\"question\":\"Why do traditional GNNs that use only positive samples face problems?\",\"answer\":\"They can suffer from over-smoothing, where node representations become less distinct with depth, reduced expressivity after aggregation, and over-squashing caused by information bottlenecks between weakly connected subgraphs.\"},{\"question\":\"How does layer-diverse negative sampling generate negative samples?\",\"answer\":\"It uses a sampling matrix within a determinantal point process to transform the candidate set into a latent space and then selectively samples negatives from that space.\"},{\"question\":\"What is the purpose of the space-squeezing technique?\",\"answer\":\"Space-squeezing enforces layer-wise diversity by removing dimensions tied to samples from later layers during sampling, which reduces the chance of repeatedly selecting redundant negatives across layers.\"}]","Layer-diverse Negative Sampling for Graph Neural Networks - Research Paper Summary | PDF",1785810662,71,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"layer-diverse-negative-sampling-for-graph-neural-networks-research-paper-summary","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/layer-diverse-negative-sampling-for-graph-neural-networks-research-paper-summary/122443/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do traditional GNNs that use only positive samples face problems?","Question",{"text":75,"@type":76},"They can suffer from over-smoothing, where node representations become less distinct with depth, reduced expressivity after aggregation, and over-squashing caused by information bottlenecks between weakly connected subgraphs.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does layer-diverse negative sampling generate negative samples?",{"text":80,"@type":76},"It uses a sampling matrix within a determinantal point process to transform the candidate set into a latent space and then selectively samples negatives from that space.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the purpose of the space-squeezing technique?",{"text":84,"@type":76},"Space-squeezing enforces layer-wise diversity by removing dimensions tied to samples from later layers during sampling, which reduces the chance of repeatedly selecting redundant negatives across layers.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]