[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-147977-105":59,"doc-detail-147977-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","diva-using-application-specific-policies-to-dive-into-vector-approximations","DiVA - Using Application-Specific Policies to “Dive” Into Vector Approximations","","High-dimensional data often makes conventional tree-based access structures underperform, so approximation-based indexing with compact vector representations is widely used. Real-world workloads, however, involve skewed data and non-uniform query distributions. DiVA addresses both by selectively combining approximation with hierarchical high fan-out indexing over data sub-spaces. It self-reorganizes using application-provided policies, adapting element refinement, and restructuring/expanding around emerging “hotspots”. Experiments on real data show DiVA reduces total I/O by up to 64% versus state-of-the-art methods.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/diva-using-application-specific-policies-to-dive-into-vector-approximations/147977/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/diva-using-application-specific-policies-to-dive-into-vector-approximations/147977.png","ImageObject",300,407,{"name":92,"@type":93},"Emma Mercer","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-22","2026-08-26",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",11,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why do tree-based indexing methods become ineffective in high-dimensional spaces?","Question",{"text":113,"@type":114},"Index performance deteriorates rapidly as the number of dimensions increases, making linear scan more efficient due to the “curse of dimensionality”.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does DiVA combine approximation and hierarchical indexing?",{"text":118,"@type":114},"DiVA selectively uses approximation and organizes data sub-spaces with a high fan-out hierarchical structure, enabling progressive access to objects with increasing granularity.",{"name":120,"@type":111,"acceptedAnswer":121},"What role do application-provided policies play in DiVA?",{"text":122,"@type":114},"Policies guide DiVA’s self-organization by triggering restructuring and possible expansion as data access patterns change, especially around “hotspots” in emerging sub-spaces.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},147977,1787773184,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":46,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":130,"read_time":31,"preview_page_count":4},962084925502,"https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8","DiVA: Using Application-Speci􀀌c Policies to “Dive” Into Vector Approximations  \nKonstantinos Tsakalozos 1 , Spiros Evangelatos2 , Fotis Psallidas3 , Marcos R. Vieira4 , Vassilis J. Tsotras5 and  \nAlex Delis2  \n1 Microsoft Corp. , London, EC1N 2ST, United Kingdom  \n2 University of Athens, Athens, 15784, Greece  \n3 Columbia University, New York, NY 10027  \n4 Hitachi America Ltd. , R&D, Santa Clara, CA 95050  \n5 University of California, Riverside, CA 92521  \nEmail: [ad@di.uoa.gr](ad@di.uoa.gr)  \nIn high-dimensional data domains, the performance of conventional tree-based access structures is occasionally outperformed by simple sequential scans. To this end, the introduction of approximation-based methods helped speed-up queries by providing compact representations of stored data. Approximation methods exploit vector quantization to index data mainly presumed to follow a uniform distribution. In real-world environments however, we mostly encounter both skewed data and query distributions. To address this dual challenge, we propose DiVA that combines the selective use of an approximation approach with an indexing mechanism to organize data sub-spaces in a high fan-out hierarchical structure. Moreover, DiVA reorganizes its own elements after receiving application hints regarding data access patterns. These hints or policies trigger the restructuring and possible expansion of DiVA so as to o􀀋er 􀀌ner indexing granularity and improved access times in sub-spaces emerging as “hotspots”. The novelty of our approach lies in the self-organizing nature of DiVA driven by application-provided policies; the latter e􀀋ectively guide the re􀀌nement of DiVA’s elements as new data arrive, existing data are updated, and the nature of query workloads continually changes. An extensive experimental evaluation using real data shows that DiVA reduces up-to 64% of the total number of I/Os if compared to state-of-art methods including the VA-􀀌le, GC-tree and A-tree.  \nKeywords: Self-Organizing Indexing Methods; Accessing Skewed Multi-dimensional Data and Queries; Vector Approximation Techniques  \nThe Computer Journal, [http: // dx. doi. org/ 10. 1093/ comjnl/ bxv097](http: // dx. doi. org/ 10. 1093/ comjnl/ bxv097) , October 2015.  \n1. INTRODUCTION  \nNumerous applications in the 􀀌elds of earth and space sciences, data analysis, scienti􀀌c computing, multimedia retrieval, bio-informatics, among others, operate on multi-dimensional data [1, 2, 3, 4, 5, 6, 7] . To e􀀎ciently evaluate similarity-based queries, data objects from such domains are typically mapped into data vectors in n-dimensional spaces. To speedup query evaluation in high-dimensional and largevolume datasets, specialized index methods have been proposed [8, 9, 10, 11, 12, 13, 14]) . These indexing schemes use a set of feature vectors, representing objects in the original domain, and a distance function to perform similarity searches. The two most common  \nsimilarity search operations in such domains are the Range (i.e., “retrieve all objects within a distance from a query object”) and the k-Nearest-Neighbor (k-NN) queries (i.e., “retrieve the k-closest objects to query object”) .  \nIn high-dimensional spaces, tree-based indexing methods employed to evaluate similarity queries are ine􀀋ective [8] . This is due to the fact that index performance deteriorates rapidly as the number of dimensions increases. This renders simple linear scan an e􀀎cient approach [15, 16] . This“curse of dimensionality” problem has led to the introduction of approximation-based access methods [9, 17, 18, 19, 20, 21] whose aim is to reduce search  \n2 K. Tsakalozos, S. Evangelatos, F. Psallidas, M. Vieira, V. Tsotras, A. Delis   \ncosts by performing sequential scan on compact and approximate representations of data. In this context, vector quantization techniques, e.g. , VA-􀀌les [15], are e􀀋ective as they employ scans on approximate quantized data to partially lift the curse of dimensionality. By and large, most of the so far","cbCaih7lWKlMuyGH","https://ap.wps.com/l/cbCaih7lWKlMuyGH","pdf",427242,"English","# Introduction\n## Similarity queries in multi-dimensional data\n## Curse of dimensionality and approximation methods\n## Hierarchical partitioning with vector quantization\n## Motivation for self-organizing, policy-driven indexing","[{\"question\":\"Why do tree-based indexing methods become ineffective in high-dimensional spaces?\",\"answer\":\"Index performance deteriorates rapidly as the number of dimensions increases, making linear scan more efficient due to the “curse of dimensionality”.\"},{\"question\":\"How does DiVA combine approximation and hierarchical indexing?\",\"answer\":\"DiVA selectively uses approximation and organizes data sub-spaces with a high fan-out hierarchical structure, enabling progressive access to objects with increasing granularity.\"},{\"question\":\"What role do application-provided policies play in DiVA?\",\"answer\":\"Policies guide DiVA’s self-organization by triggering restructuring and possible expansion as data access patterns change, especially around “hotspots” in emerging sub-spaces.\"}]","DiVA - Using Application-Specific Policies to “Dive” Into Vector Approximations | PDF"]