[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85527-en":3,"doc-seo-85527-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85527,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","PiPNN Ultra-Scalable Graph-Based Nearest Neighbor Indexing","Fast approximate nearest neighbor search systems based on graph indexes deliver state-of-the-art query performance, yet pay an unacceptable cost in construction time because they rely on random-access-heavy beam searches. This paper presents PiPNN (Pick-in-Partitions Nearest Neighbors), an ultra-scalable graph construction approach that removes the construction search bottleneck. Its HashPrune online pruning algorithm maintains sparse edge sets online, partitions data into overlapping subproblems, and streams edges while bounding memory. Experiments show up to 11.6× and 12.9× faster builds than Vamana and HNSW, respectively, enabling billion-scale indexing in under 20 minutes.","PiPNN: Ultra-Scalable Graph-Based Nearest Neighbor Indexing  \nTobias Rubel  \nUMD  \nCollege Park, Maryland [trubel@umd.edu](trubel@umd.edu)  \nLars Gottesbüren  \nGoogle Research Zürich, Switzerland  \nRichard Wen  \nUMD College Park, Maryland [rwen1@umd.edu](rwen1@umd.edu)  \nRajesh Jayaram  \nGoogle Research New York City, New York  \nLaxman Dhulipala  \nUMD and Google Research College Park, Maryland [laxman@umd.edu](laxman@umd.edu)  \nJakub Łącki  \nGoogle Research New York City, New York  \narXiv :2602 .21247v3 [ cs .DB] 10 Jul 2026  \nABSTRACT  \nThe fastest indexes for Approximate Nearest Neighbor Search (ANNS) today are also the slowest to build: graph-based methods like HNSW and Vamana achieve state-of-the-art query performance but have prohibitively large construction times due to relying on random-access-heavy beam searches. In this paper, we introduce PiPNN (Pick-in-Partitions Nearest Neighbors), an ultra-scalable graph construction algorithm that avoids this “search bottleneck”that existing graph-based methods suffer from.  \nPiPNN’s core innovation is HashPrune, a novel online pruning algorithm which dynamically maintains sparse collections of edges. HashPrune enables PiPNN to partition the dataset into overlapping sub-problems, efficiently perform bulk distance comparisons via dense matrix multiplication kernels, and stream a subset of the edges into HashPrune. HashPrune guarantees bounded memory during index construction which permits PiPNN to build higher quality indices without the use of extra intermediate memory.  \nOur extensive experimental study demonstrates that PiPNN builds state-of-the-art indexes up to 11. 6× faster than Vamana (DiskANN) and up to 12. 9× faster than HNSW. We show that these improvements extend to downstream tasks, yielding speedups of up to 1. 9× for approximate 􀀺-NN graph construction. PiPNN is significantly more scalable than recent algorithms for fast graph construction. PiPNN builds indexes at least 19. 1× faster than MIRAGE and 17. 3× than FastKCNA while producing indexes that achieve significantly higher query throughput. PiPNN enables us to build, for the first time, high-quality ANN indexes on billion-scale datasets in under 20 minutes using a single multicore machine.  \n1 INTRODUCTION  \nHigh-dimensional vector embeddings are a fundamental datatype used in modern search, information retrieval, classification, and recommendation applications. For example, vector embeddings are the backbone of a diverse set of applications including entity resolution [28, 46], retrieval-augmented generation (RAG) systems for generating context for large language models (LLMs) [17, 27], and recommendation systems [31, 36, 42] . A fundamental problem in these applications is nearest neighbor search: given a query vector (point), find its nearest neighbors from a large set of points according to some distance function (e.g., 􀀡2 or inner product distance) .  \n PiPNN (1 Replica)  HCNNG  Vamana (1 Pass)  HNSW  \nSpeedup over HNSW  \nOpenAI Wikipedia Cohere  \nBIGANN-1BMSSPACEV-1B  \nMSTURING-1BDEEP-1B  \nFigure 1: Build time speedup compared to HNSW on six benchmarks, including billion-scale inputs from big-ann-benchmarks.  \nDue to the notorious difficulty of finding the exact nearest neighbors in high-dimensional spaces [10] and the fact that real-world applications typically tolerate small errors, modern embedding-based applications leverage approximate nearest-neighbor search (ANNS). In recent years, graph-based indexing methods such as HNSW [29], Vamana [24, 26], and NSG [16] have become the standard for highrecall and low-latency ANNS. These methods add edges between nearby points along with carefully chosen long-range edges for navigability, enabling fast queries using beam search.  \nAlthough graph-based indexing methods are the fastest for querying, they are also extremely slow to build, and their high construction cost undermines their utility in real-world applications where index build time is just as important ","cbCaicJ9S0Luy32G","https://ap.wps.com/l/cbCaicJ9S0Luy32G","pdf",3421153,3,1,18,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does PiPNN address in graph-based approximate nearest neighbor indexing?\",\"answer\":\"Graph-based ANNS methods like HNSW and Vamana are fast at querying but extremely slow to build due to random-access-heavy beam searches. PiPNN targets this construction-time “search bottleneck.”\"},{\"question\":\"How does PiPNN’s HashPrune algorithm improve index construction efficiency?\",\"answer\":\"HashPrune is an online pruning method that dynamically maintains sparse edge collections. It supports partitioning into overlapping subproblems, efficient bulk distance comparisons, and streaming edges into the pruning process while bounding memory usage.\"},{\"question\":\"What performance improvements does the paper report for PiPNN compared with prior methods?\",\"answer\":\"The experiments show PiPNN builds state-of-the-art indexes up to 11.6× faster than Vamana and up to 12.9× faster than HNSW, with downstream speedups and higher query throughput. The method can build high-quality ANN indexes on billion-scale datasets in under 20 minutes on a single multicore machine.\"}]",1784204184,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"pipnn-ultra-scalable-graph-based-nearest-neighbor-indexing","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/pipnn-ultra-scalable-graph-based-nearest-neighbor-indexing/85527/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does PiPNN address in graph-based approximate nearest neighbor indexing?","Question",{"text":75,"@type":76},"Graph-based ANNS methods like HNSW and Vamana are fast at querying but extremely slow to build due to random-access-heavy beam searches. PiPNN targets this construction-time “search bottleneck.”","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PiPNN’s HashPrune algorithm improve index construction efficiency?",{"text":80,"@type":76},"HashPrune is an online pruning method that dynamically maintains sparse edge collections. It supports partitioning into overlapping subproblems, efficient bulk distance comparisons, and streaming edges into the pruning process while bounding memory usage.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance improvements does the paper report for PiPNN compared with prior methods?",{"text":84,"@type":76},"The experiments show PiPNN builds state-of-the-art indexes up to 11.6× faster than Vamana and up to 12.9× faster than HNSW, with downstream speedups and higher query throughput. The method can build high-quality ANN indexes on billion-scale datasets in under 20 minutes on a single multicore machine.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]